2024-02-05T06:14:32.584 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-05T06:14:32.584 DEBUG:teuthology.run:Teuthology command: teuthology --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} --verbose --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689 --name gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi -- /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/orig.config.yaml 2024-02-05T06:14:32.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T06:14:32.689 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689 branch: wip-guits-testing-3-2024-02-04-2254 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7546689' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-3-2024-02-04-2254 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-3-2024-02-04-2254 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 7317 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-3-2024-02-04-2254 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMgdWwhNXncLNrk8F7H2F7c+6jNJEs7cdStV5ch07938MyRUC+au+l4ikI9mmlLmK97hL546eK3EzJjTX67qcTU= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKtEh0Ft4nT4zvD/RsICi5M88SxKUHRSz7dGoB2If7JcoHK6aacCPMeP0fLdm0KPBpG97CPdNILEFv9IqdblsyA= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3CGnu0UDr1c/RqYaNfvC2ZQaUGg4LVpYRmFOPD5Fr8Ia3YA7pY3Mt6xpEQX1kCKvaU+2BNnjUQK11J4dgcyLI= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-05_05:57:01 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-05T06:14:32.689 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa; will attempt to use it 2024-02-05T06:14:32.690 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa/tasks 2024-02-05T06:14:32.690 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-05T06:14:32.691 INFO:teuthology.task.internal:Checking packages... 2024-02-05T06:14:32.716 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '164d3be4a527be775f4d3c343ea591122f1b8921' 2024-02-05T06:14:32.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-05T06:14:32.716 INFO:teuthology.packaging:ref: None 2024-02-05T06:14:32.716 INFO:teuthology.packaging:tag: None 2024-02-05T06:14:32.716 INFO:teuthology.packaging:branch: wip-guits-testing-3-2024-02-04-2254 2024-02-05T06:14:32.716 INFO:teuthology.packaging:sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:14:32.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-3-2024-02-04-2254 2024-02-05T06:14:32.851 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1241.g164d3be4 2024-02-05T06:14:32.853 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-05T06:14:32.859 INFO:teuthology.task.internal:no buildpackages task found 2024-02-05T06:14:32.859 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-05T06:14:32.871 INFO:teuthology.task.internal:Saving configuration 2024-02-05T06:14:32.882 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-05T06:14:32.887 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T06:14:32.912 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.758647', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMgdWwhNXncLNrk8F7H2F7c+6jNJEs7cdStV5ch07938MyRUC+au+l4ikI9mmlLmK97hL546eK3EzJjTX67qcTU='} 2024-02-05T06:14:32.935 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.762014', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKtEh0Ft4nT4zvD/RsICi5M88SxKUHRSz7dGoB2If7JcoHK6aacCPMeP0fLdm0KPBpG97CPdNILEFv9IqdblsyA='} 2024-02-05T06:14:32.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.760627', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3CGnu0UDr1c/RqYaNfvC2ZQaUGg4LVpYRmFOPD5Fr8Ia3YA7pY3Mt6xpEQX1kCKvaU+2BNnjUQK11J4dgcyLI='} 2024-02-05T06:14:32.959 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-05T06:14:32.965 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-05T06:14:32.966 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-05T06:14:32.966 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-05T06:14:32.966 INFO:teuthology.run_tasks:Running task console_log... 2024-02-05T06:14:33.078 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f812821ff70>, signals=[15]) 2024-02-05T06:14:33.079 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-05T06:14:33.085 INFO:teuthology.task.internal:Opening connections... 2024-02-05T06:14:33.085 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-02-05T06:14:33.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:14:33.180 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-02-05T06:14:33.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:14:33.272 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-02-05T06:14:33.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:14:33.364 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-05T06:14:33.370 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-02-05T06:14:33.389 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-02-05T06:14:33.389 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-02-05T06:14:33.446 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-02-05T06:14:33.446 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-02-05T06:14:33.446 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-02-05T06:14:33.446 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-02-05T06:14:33.446 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T06:14:33.447 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T06:14:33.448 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-02-05T06:14:33.476 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-02-05T06:14:33.493 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-02-05T06:14:33.493 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-02-05T06:14:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-02-05T06:14:33.551 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T06:14:33.551 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-02-05T06:14:33.551 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T06:14:33.551 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T06:14:33.551 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T06:14:33.551 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-02-05T06:14:33.576 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-02-05T06:14:33.593 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-02-05T06:14:33.593 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T06:14:33.651 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-02-05T06:14:33.652 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T06:14:33.652 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T06:14:33.652 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T06:14:33.652 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-02-05T06:14:33.675 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-05T06:14:33.685 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-05T06:14:33.690 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-05T06:14:33.690 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-02-05T06:14:33.692 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-02-05T06:14:33.695 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-02-05T06:14:33.710 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-05T06:14:33.716 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-05T06:14:33.716 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-02-05T06:14:33.749 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-02-05T06:14:33.751 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-02-05T06:14:34.004 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-05T06:14:34.081 INFO:teuthology.run_tasks:Running task kernel... 2024-02-05T06:14:34.102 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-05T06:14:34.103 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-05T06:14:34.103 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T06:14:34.104 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T06:14:34.105 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T06:14:34.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:14:34.123 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-02-05T06:14:34.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:14:34.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:14:34.126 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-02-05T06:14:34.127 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-02-05T06:14:34.184 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-361.el9.x86_64 2024-02-05T06:14:34.184 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-361.el9.x86_64 2024-02-05T06:14:34.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-02-05T06:14:34.188 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-361.el9.x86_64 2024-02-05T06:14:34.188 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-361.el9.x86_64 2024-02-05T06:14:34.188 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-361.el9.x86_64 2024-02-05T06:14:34.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-02-05T06:14:34.189 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.14.0-361.el9.x86_64 2024-02-05T06:14:34.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum install -y kernel 2024-02-05T06:14:36.263 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 15 MB/s | 6.6 MB 00:00 2024-02-05T06:14:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 11 MB/s | 6.6 MB 00:00 2024-02-05T06:14:36.466 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 11 MB/s | 6.6 MB 00:00 2024-02-05T06:14:37.628 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-02-05T06:14:37.833 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-05T06:14:38.040 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-02-05T06:14:38.827 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-02-05T06:14:40.090 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-02-05T06:14:41.236 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-02-05T06:14:42.000 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-05T06:14:43.063 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-02-05T06:14:45.236 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.9 MB 00:00 2024-02-05T06:14:46.729 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-02-05T06:14:48.283 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 5.9 MB 00:02 2024-02-05T06:14:48.674 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-05T06:14:49.759 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-02-05T06:14:51.635 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-05T06:14:53.588 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-02-05T06:14:55.819 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T06:14:55.819 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T06:14:55.920 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-05T06:14:55.931 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:14:55.931 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-02-05T06:14:55.932 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:14:55.933 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 99 M 2024-02-05T06:14:55.933 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 122 M 2024-02-05T06:14:55.933 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-05T06:14:56.272 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-02-05T06:14:56.443 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-02-05T06:14:56.789 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-02-05T06:14:57.423 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): kernel-modules-core-5.14.0-416.el9.x86_6 29 MB/s | 33 MB 00:01 2024-02-05T06:14:57.839 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-05T06:14:57.843 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:14:57.843 INFO:teuthology.orchestra.run.smithi142.stdout:Total 52 MB/s | 99 MB 00:01 2024-02-05T06:14:57.843 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-05T06:14:58.217 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-05T06:14:58.217 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-05T06:14:58.633 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T06:14:58.633 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T06:14:58.727 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-05T06:14:58.737 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:14:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-02-05T06:14:58.737 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-02-05T06:14:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:14:58.739 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 99 M 2024-02-05T06:14:58.739 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 122 M 2024-02-05T06:14:58.739 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-05T06:14:59.069 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-02-05T06:14:59.280 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-05T06:14:59.281 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-05T06:14:59.553 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 26 MB/s | 21 MB 00:00 2024-02-05T06:14:59.872 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-02-05T06:15:00.186 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-modules-core-5.14.0-416.el9.x86_6 30 MB/s | 33 MB 00:01 2024-02-05T06:15:00.695 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-05T06:15:00.698 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:15:00.699 INFO:teuthology.orchestra.run.smithi196.stdout:Total 51 MB/s | 99 MB 00:01 2024-02-05T06:15:00.699 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-05T06:15:01.067 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-05T06:15:01.067 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-05T06:15:01.669 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-05T06:15:02.112 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-05T06:15:02.112 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-05T06:15:02.353 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:15:02.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:04.508 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-05T06:15:05.240 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:15:05.242 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:05.361 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:05.405 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:05.439 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 5.9 MB 00:02 2024-02-05T06:15:07.415 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2024-02-05T06:15:08.104 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:10.122 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:10.457 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:12.968 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:13.167 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:15.167 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:17.887 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:38.683 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 668 kB/s | 20 MB 00:31 2024-02-05T06:15:43.563 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-05T06:15:44.573 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:44.574 INFO:teuthology.orchestra.run.smithi142.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T06:15:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:15:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:45.038 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:45.038 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:15:45.038 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:45.038 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T06:15:45.486 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T06:15:45.487 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:15:45.487 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-05T06:15:45.770 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T06:15:45.770 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T06:15:45.870 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-02-05T06:15:45.876 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:15:45.876 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-02-05T06:15:45.876 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:15:45.876 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-02-05T06:15:45.876 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-02-05T06:15:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:15:45.878 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 99 M 2024-02-05T06:15:45.878 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 122 M 2024-02-05T06:15:45.878 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-02-05T06:15:46.001 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:15:46.146 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:46.147 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:15:46.147 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T06:15:46.147 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T06:15:46.147 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T06:15:46.147 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-05T06:15:46.148 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T06:15:46.148 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:15:46.223 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 17 MB/s | 6.0 MB 00:00 2024-02-05T06:15:46.255 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:46.256 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:15:46.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-02-05T06:15:46.291 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-02-05T06:15:46.292 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-02-05T06:15:46.293 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-02-05T06:15:46.293 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-02-05T06:15:46.293 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-02-05T06:15:46.293 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T06:15:46.293 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-02-05T06:15:46.294 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-02-05T06:15:46.294 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-02-05T06:15:46.294 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T06:15:46.294 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T06:15:46.295 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T06:15:46.295 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T06:15:46.295 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-02-05T06:15:46.296 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-02-05T06:15:46.296 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T06:15:46.296 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-02-05T06:15:46.296 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-02-05T06:15:46.297 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:15:46.297 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T06:15:46.297 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T06:15:46.297 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T06:15:46.297 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-02-05T06:15:46.298 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:15:46.298 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T06:15:46.301 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T06:15:46.301 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 5.14.0-416.el9.x86_64 2024-02-05T06:15:46.301 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T06:15:46.936 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-02-05T06:15:46.948 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:46.948 INFO:teuthology.orchestra.run.smithi196.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T06:15:46.948 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:15:47.007 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 18 MB/s | 21 MB 00:01 2024-02-05T06:15:47.016 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:47.382 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 26 MB/s | 39 MB 00:01 2024-02-05T06:15:47.496 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:47.496 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:15:47.496 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:47.496 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:15:47.840 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-core-5.14.0-416.el9.x86_6 21 MB/s | 33 MB 00:01 2024-02-05T06:15:47.844 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:15:47.844 INFO:teuthology.orchestra.run.smithi122.stdout:Total 50 MB/s | 99 MB 00:01 2024-02-05T06:15:47.844 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-02-05T06:15:47.969 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:15:47.969 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:15:47.969 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:15:47.970 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-05T06:15:48.253 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-02-05T06:15:48.253 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-02-05T06:15:48.308 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T06:15:48.337 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-02-05T06:15:48.339 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-02-05T06:15:48.357 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.9oREa6zyxT 2024-02-05T06:15:48.358 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9oREa6zyxT 2024-02-05T06:15:48.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.9oREa6zyxT 2024-02-05T06:15:48.485 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:15:48.568 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.9oREa6zyxT is 6KB 2024-02-05T06:15:48.617 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.9oREa6zyxT 2024-02-05T06:15:48.618 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:48.620 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:15:48.620 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T06:15:48.620 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T06:15:48.620 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T06:15:48.621 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-05T06:15:48.621 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T06:15:48.621 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:15:48.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T06:15:48.705 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T06:15:48.705 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T06:15:48.705 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T06:15:48.705 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T06:15:48.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T06:15:48.728 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:15:48.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:15:48.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-02-05T06:15:48.762 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 68.el9 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8264254 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-02-05T06:15:48.763 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:15:48.764 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T06:15:48.765 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T06:15:48.765 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-416.el9.x86_64 2024-02-05T06:15:48.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T06:15:48.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-02-05T06:15:49.258 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-02-05T06:15:49.259 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-02-05T06:15:49.394 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-02-05T06:15:50.800 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T06:15:50.836 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-02-05T06:15:50.839 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-02-05T06:15:50.860 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.XUeRSBdtZk 2024-02-05T06:15:50.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XUeRSBdtZk 2024-02-05T06:15:50.930 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.XUeRSBdtZk 2024-02-05T06:15:51.069 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.XUeRSBdtZk is 6KB 2024-02-05T06:15:51.118 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.XUeRSBdtZk 2024-02-05T06:15:51.132 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T06:15:51.199 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T06:15:51.199 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T06:15:51.199 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T06:15:51.199 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T06:15:51.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T06:15:51.334 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-02-05T06:15:51.651 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-02-05T06:15:52.309 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:15:52.493 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:15:55.391 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:16:00.079 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:16:00.313 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:05.030 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:18.872 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-05T06:16:18.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-05T06:16:18.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:16:21.337 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-05T06:16:21.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-05T06:16:21.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:16:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:34.188 INFO:teuthology.orchestra.run.smithi122.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T06:16:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:16:34.258 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T06:16:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T06:16:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T06:16:35.112 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T06:16:35.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:16:35.112 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-02-05T06:16:35.112 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.112 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.113 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.113 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:16:35.113 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-02-05T06:16:35.618 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:16:35.766 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.766 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:16:35.766 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T06:16:35.766 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T06:16:35.766 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T06:16:35.766 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-05T06:16:35.767 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T06:16:35.767 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T06:16:35.874 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T06:16:35.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T06:16:35.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-02-05T06:16:35.907 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 68.el9 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8264254 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-02-05T06:16:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:16:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T06:16:35.910 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T06:16:35.910 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-416.el9.x86_64 2024-02-05T06:16:35.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T06:16:36.519 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-02-05T06:16:37.986 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T06:16:38.016 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-02-05T06:16:38.018 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-02-05T06:16:38.033 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.m3AnX91zYg 2024-02-05T06:16:38.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m3AnX91zYg 2024-02-05T06:16:38.101 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.m3AnX91zYg 2024-02-05T06:16:38.247 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.m3AnX91zYg is 6KB 2024-02-05T06:16:38.295 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.m3AnX91zYg 2024-02-05T06:16:38.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T06:16:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T06:16:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T06:16:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T06:16:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T06:16:38.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T06:16:38.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-02-05T06:17:08.552 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-05T06:17:08.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-02-05T06:17:08.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:17:18.884 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T06:17:21.342 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T06:17:27.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-05T06:17:27.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:17:29.278 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-02-05T06:17:29.838 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-05T06:17:29.838 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-416.el9.x86_64"... 2024-02-05T06:17:29.838 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-02-05T06:17:29.852 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-416.el9.x86_64 2024-02-05T06:17:29.852 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T06:17:29.852 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T06:17:29.853 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T06:17:30.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-05T06:17:30.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:17:30.738 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-02-05T06:17:30.853 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-05T06:17:30.853 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T06:17:30.960 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-02-05T06:17:30.990 DEBUG:teuthology.parallel:result is None 2024-02-05T06:17:31.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-05T06:17:31.143 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-416.el9.x86_64"... 2024-02-05T06:17:31.143 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-02-05T06:17:31.157 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-416.el9.x86_64 2024-02-05T06:17:31.158 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T06:17:31.158 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T06:17:31.158 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T06:17:32.159 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-05T06:17:32.159 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T06:17:32.235 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-02-05T06:17:32.276 DEBUG:teuthology.parallel:result is None 2024-02-05T06:18:08.557 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T06:18:17.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-02-05T06:18:17.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:18:21.063 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-02-05T06:18:21.537 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-02-05T06:18:21.537 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-416.el9.x86_64"... 2024-02-05T06:18:21.538 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-02-05T06:18:21.552 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-416.el9.x86_64 2024-02-05T06:18:21.552 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T06:18:21.552 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T06:18:21.552 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T06:18:22.553 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-05T06:18:22.553 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T06:18:22.627 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-02-05T06:18:22.680 DEBUG:teuthology.parallel:result is None 2024-02-05T06:18:22.680 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-05T06:18:22.688 INFO:teuthology.task.internal:Creating test directory... 2024-02-05T06:18:22.689 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T06:18:22.691 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T06:18:22.694 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T06:18:22.713 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-05T06:18:22.719 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-05T06:18:22.729 INFO:teuthology.task.internal:Creating archive directory... 2024-02-05T06:18:22.729 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T06:18:22.746 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T06:18:22.753 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T06:18:22.826 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-05T06:18:22.832 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-05T06:18:22.832 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T06:18:22.835 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T06:18:22.837 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T06:18:22.872 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.874 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.877 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.886 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.888 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.891 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T06:18:22.893 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-05T06:18:22.899 INFO:teuthology.task.internal:Configuring sudo... 2024-02-05T06:18:22.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T06:18:22.931 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T06:18:22.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T06:18:22.964 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-05T06:18:22.974 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-05T06:18:22.974 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T06:18:22.999 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T06:18:23.005 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T06:18:23.021 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.094 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.140 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.264 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:18:23.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T06:18:23.336 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.381 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.448 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.536 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:18:23.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T06:18:23.607 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.654 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T06:18:23.715 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.759 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T06:18:23.803 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:18:23.803 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T06:18:23.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-02-05T06:18:23.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-02-05T06:18:23.877 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-02-05T06:18:23.923 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:18:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:18:23.971 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:18:24.332 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-05T06:18:24.340 INFO:teuthology.task.internal:Starting timer... 2024-02-05T06:18:24.340 INFO:teuthology.run_tasks:Running task pcp... 2024-02-05T06:18:24.349 INFO:teuthology.run_tasks:Running task selinux... 2024-02-05T06:18:24.357 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-05T06:18:24.358 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-02-05T06:18:24.421 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-02-05T06:18:24.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-02-05T06:18:24.470 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-02-05T06:18:24.471 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-02-05T06:18:24.540 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-02-05T06:18:24.542 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-05T06:18:24.542 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-02-05T06:18:24.566 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-02-05T06:18:24.566 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-02-05T06:18:24.587 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-02-05T06:18:24.587 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-02-05T06:18:24.608 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-02-05T06:18:24.608 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-02-05T06:18:24.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:18:24.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:18:24.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:18:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:18:24.667 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:18:24.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:18:24.695 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-05T06:18:24.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-02-05T06:18:24.734 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-02-05T06:18:24.771 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-02-05T06:18:24.801 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-05T06:18:24.811 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-05T06:18:24.828 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-05T06:18:24.830 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-05T06:22:31.509 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-02-05T06:22:31.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-02-05T06:22:31.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:22:31.608 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-02-05T06:22:31.682 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-02-05T06:22:31.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-05T06:22:31.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:22:31.772 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-02-05T06:22:31.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-05T06:22:31.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-05T06:22:31.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T06:22:31.945 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-02-05T06:22:32.023 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-02-05T06:22:32.023 INFO:teuthology.run_tasks:Running task clock... 2024-02-05T06:22:32.033 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-05T06:22:32.034 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T06:22:32.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:22:32.037 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T06:22:32.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:22:32.039 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T06:22:32.039 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:22:32.069 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T06:22:32.071 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T06:22:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T06:22:32.089 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T06:22:32.093 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T06:22:32.110 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T06:22:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-02-05T06:22:32.120 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-02-05T06:22:32.132 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-02-05T06:22:32.134 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-02-05T06:22:32.139 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-02-05T06:22:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T06:22:32.151 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T06:22:32.153 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-02-05T06:22:32.166 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T06:22:32.168 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T06:22:32.170 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T06:22:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T06:22:32.217 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:22:32.218 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:22:32.221 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:22:32.221 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-02-05T06:22:32.221 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.221 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.222 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.222 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:22:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-05T06:22:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.249 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.253 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:22:32.254 INFO:teuthology.run_tasks:Running task pexec... 2024-02-05T06:22:32.264 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-05T06:22:32.264 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi122.front.sepia.ceph.com 2024-02-05T06:22:32.264 DEBUG:teuthology.orchestra.run.smithi122:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T06:22:32.265 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2024-02-05T06:22:32.265 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T06:22:32.265 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-02-05T06:22:32.265 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T06:22:32.847 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:49 ago on Mon 05 Feb 2024 06:21:43 AM UTC. 2024-02-05T06:22:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:49 ago on Mon 05 Feb 2024 06:21:43 AM UTC. 2024-02-05T06:22:32.878 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:49 ago on Mon 05 Feb 2024 06:21:43 AM UTC. 2024-02-05T06:22:32.997 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-02-05T06:22:33.003 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-05T06:22:33.008 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-02-05T06:22:33.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:22:33.010 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 3.1 M 2024-02-05T06:22:33.010 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 9.8 M 2024-02-05T06:22:33.010 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-02-05T06:22:33.014 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:22:33.014 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-02-05T06:22:33.014 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:22:33.014 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-02-05T06:22:33.014 INFO:teuthology.orchestra.run.smithi142.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 3.1 M 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 9.8 M 2024-02-05T06:22:33.015 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-05T06:22:33.028 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-02-05T06:22:33.043 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:22:33.044 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.1 M 2024-02-05T06:22:33.044 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 9.8 M 2024-02-05T06:22:33.044 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-02-05T06:22:33.549 INFO:teuthology.orchestra.run.smithi142.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.9 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:22:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:Total 5.8 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.550 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-05T06:22:33.555 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-05T06:22:33.555 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-05T06:22:33.562 INFO:teuthology.orchestra.run.smithi122.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.7 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.563 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:22:33.563 INFO:teuthology.orchestra.run.smithi122.stdout:Total 5.6 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.563 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-02-05T06:22:33.565 INFO:teuthology.orchestra.run.smithi196.stdout:runc-1.1.12-1.el9.x86_64.rpm 6.0 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.566 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:22:33.566 INFO:teuthology.orchestra.run.smithi196.stdout:Total 6.0 MB/s | 3.1 MB 00:00 2024-02-05T06:22:33.567 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-02-05T06:22:33.569 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-02-05T06:22:33.569 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-02-05T06:22:33.572 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-02-05T06:22:33.572 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-02-05T06:22:33.626 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-05T06:22:33.626 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-05T06:22:33.640 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-02-05T06:22:33.641 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-02-05T06:22:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-02-05T06:22:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-02-05T06:22:33.831 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-05T06:22:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-02-05T06:22:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-02-05T06:22:33.932 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:33.935 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.493 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.507 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.519 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.813 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.813 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:22:34.813 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-05T06:22:34.813 INFO:teuthology.orchestra.run.smithi142.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T06:22:34.814 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-05T06:22:34.814 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:22:34.834 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:22:34.857 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-02-05T06:22:35.049 DEBUG:teuthology.parallel:result is None 2024-02-05T06:22:35.137 DEBUG:teuthology.parallel:result is None 2024-02-05T06:22:35.146 DEBUG:teuthology.parallel:result is None 2024-02-05T06:22:35.146 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-05T06:22:35.157 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-05T06:22:35.157 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:22:35.158 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:35.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T06:22:35.173 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:22:30.499544474 +0000 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:08.500045594 +0000 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:08.500045594 +0000 2024-02-05T06:22:35.227 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:08.500045594 +0000 2024-02-05T06:22:35.228 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T06:22:35.292 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:22:35.292 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:22:35.292 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000190203 s, 2.7 MB/s 2024-02-05T06:22:35.294 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T06:22:35.349 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-02-05T06:22:35.403 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:22:30.499544474 +0000 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:08.859037417 +0000 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:08.859037417 +0000 2024-02-05T06:22:35.404 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:08.859037417 +0000 2024-02-05T06:22:35.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T06:22:35.470 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:22:35.470 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:22:35.470 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000207604 s, 2.5 MB/s 2024-02-05T06:22:35.471 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T06:22:35.526 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-02-05T06:22:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:22:30.499544474 +0000 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:09.191029855 +0000 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:09.191029855 +0000 2024-02-05T06:22:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:09.191029855 +0000 2024-02-05T06:22:35.581 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T06:22:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:22:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:22:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000205414 s, 2.5 MB/s 2024-02-05T06:22:35.647 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T06:22:35.702 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:22:30.499544474 +0000 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:09.531022110 +0000 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:09.531022110 +0000 2024-02-05T06:22:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:09.531022110 +0000 2024-02-05T06:22:35.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T06:22:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:22:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:22:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000212209 s, 2.4 MB/s 2024-02-05T06:22:35.824 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T06:22:35.880 DEBUG:teuthology.orchestra.run.smithi122:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-05T06:22:36.020 INFO:teuthology.orchestra.run.smithi122.stdout:loop 2024-02-05T06:22:36.024 INFO:tasks.nvme_loop:Connecting nvme_loop smithi122:/dev/vg_nvme/lv_1... 2024-02-05T06:22:36.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-05T06:22:36.061 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-02-05T06:22:36.098 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_11 2024-02-05T06:22:36.135 INFO:tasks.nvme_loop:Connecting nvme_loop smithi122:/dev/vg_nvme/lv_2... 2024-02-05T06:22:36.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-05T06:22:36.215 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-02-05T06:22:36.254 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_21 2024-02-05T06:22:36.282 INFO:tasks.nvme_loop:Connecting nvme_loop smithi122:/dev/vg_nvme/lv_3... 2024-02-05T06:22:36.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-05T06:22:36.361 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-02-05T06:22:36.398 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_31 2024-02-05T06:22:36.439 INFO:tasks.nvme_loop:Connecting nvme_loop smithi122:/dev/vg_nvme/lv_4... 2024-02-05T06:22:36.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-05T06:22:36.517 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-02-05T06:22:36.554 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_41 2024-02-05T06:22:36.582 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:22:36.582 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:36.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo nvme list 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/nvme0n1 PHFT6204010U400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101F0 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/nvme1n1 12473900da27e5869e4c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/nvme2n1 e685cf3c172ef9c7b59f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/nvme3n1 771524f7e64fc963e6a0 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:36.706 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/nvme4n1 a04bfe6ff2b181ad27e8 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:36.706 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:22:36.707 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:22:36.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/scratch_devs 2024-02-05T06:22:36.774 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:22:36.774 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:36.793 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T06:22:36.793 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-02-05T06:22:36.850 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T06:22:36.850 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:22:30.524148410 +0000 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:09.058553046 +0000 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:09.058553046 +0000 2024-02-05T06:22:36.851 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:09.058553046 +0000 2024-02-05T06:22:36.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T06:22:36.920 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:22:36.920 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:22:36.920 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000313881 s, 1.6 MB/s 2024-02-05T06:22:36.921 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T06:22:36.979 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:22:30.524148410 +0000 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:09.404546524 +0000 2024-02-05T06:22:37.036 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:09.404546524 +0000 2024-02-05T06:22:37.037 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:09.404546524 +0000 2024-02-05T06:22:37.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T06:22:37.103 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:22:37.103 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:22:37.104 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000195076 s, 2.6 MB/s 2024-02-05T06:22:37.104 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T06:22:37.161 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-02-05T06:22:37.216 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T06:22:37.216 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:37.216 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:22:30.524148410 +0000 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:09.737540247 +0000 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:09.737540247 +0000 2024-02-05T06:22:37.217 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:09.737540247 +0000 2024-02-05T06:22:37.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T06:22:37.283 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:22:37.283 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:22:37.283 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000194839 s, 2.6 MB/s 2024-02-05T06:22:37.285 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T06:22:37.341 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-02-05T06:22:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:22:30.524148410 +0000 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:10.075533876 +0000 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:10.075533876 +0000 2024-02-05T06:22:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:10.075533876 +0000 2024-02-05T06:22:37.396 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T06:22:37.461 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:22:37.462 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:22:37.462 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00018878 s, 2.7 MB/s 2024-02-05T06:22:37.463 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T06:22:37.518 DEBUG:teuthology.orchestra.run.smithi142:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-05T06:22:37.658 INFO:teuthology.orchestra.run.smithi142.stdout:loop 2024-02-05T06:22:37.660 INFO:tasks.nvme_loop:Connecting nvme_loop smithi142:/dev/vg_nvme/lv_1... 2024-02-05T06:22:37.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-05T06:22:37.697 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-02-05T06:22:37.735 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/vg_nvme/lv_11 2024-02-05T06:22:37.764 INFO:tasks.nvme_loop:Connecting nvme_loop smithi142:/dev/vg_nvme/lv_2... 2024-02-05T06:22:37.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-05T06:22:37.842 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-02-05T06:22:37.882 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/vg_nvme/lv_21 2024-02-05T06:22:37.910 INFO:tasks.nvme_loop:Connecting nvme_loop smithi142:/dev/vg_nvme/lv_3... 2024-02-05T06:22:37.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-05T06:22:37.989 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-02-05T06:22:38.026 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/vg_nvme/lv_31 2024-02-05T06:22:38.061 INFO:tasks.nvme_loop:Connecting nvme_loop smithi142:/dev/vg_nvme/lv_4... 2024-02-05T06:22:38.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-05T06:22:38.139 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-02-05T06:22:38.176 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/vg_nvme/lv_41 2024-02-05T06:22:38.204 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:22:38.205 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:38.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo nvme list 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/nvme0n1 CVFT6303009Y400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/nvme1n1 78345f66dd76eaa31772 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/nvme2n1 312a3bc237f8cd5e1de4 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/nvme3n1 93639b3d818b083c4439 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:38.326 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/nvme4n1 d56b3f5a1e7ac519be6a Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:38.327 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:22:38.327 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:22:38.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/scratch_devs 2024-02-05T06:22:38.393 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:22:38.393 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:38.412 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T06:22:38.413 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:22:30.537463792 +0000 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:09.520920437 +0000 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:09.520920437 +0000 2024-02-05T06:22:38.470 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:09.520920437 +0000 2024-02-05T06:22:38.471 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T06:22:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:22:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:22:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000248981 s, 2.1 MB/s 2024-02-05T06:22:38.542 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T06:22:38.600 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-02-05T06:22:38.656 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T06:22:38.656 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:38.656 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:22:30.537463792 +0000 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:09.855913159 +0000 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:09.855913159 +0000 2024-02-05T06:22:38.657 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:09.855913159 +0000 2024-02-05T06:22:38.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T06:22:38.724 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:22:38.724 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:22:38.724 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000215304 s, 2.4 MB/s 2024-02-05T06:22:38.725 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T06:22:38.782 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-02-05T06:22:38.838 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T06:22:38.838 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:38.838 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-05T06:22:38.838 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:38.838 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:38.839 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:22:30.537463792 +0000 2024-02-05T06:22:38.839 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:10.211905424 +0000 2024-02-05T06:22:38.839 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:10.211905424 +0000 2024-02-05T06:22:38.839 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:10.211905424 +0000 2024-02-05T06:22:38.839 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T06:22:38.904 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:22:38.904 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:22:38.904 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000191807 s, 2.7 MB/s 2024-02-05T06:22:38.906 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T06:22:38.962 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:22:30.537463792 +0000 2024-02-05T06:22:39.017 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:10.541898254 +0000 2024-02-05T06:22:39.018 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:10.541898254 +0000 2024-02-05T06:22:39.018 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:10.541898254 +0000 2024-02-05T06:22:39.018 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T06:22:39.082 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:22:39.083 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:22:39.083 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000248446 s, 2.1 MB/s 2024-02-05T06:22:39.084 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T06:22:39.139 DEBUG:teuthology.orchestra.run.smithi196:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-05T06:22:39.280 INFO:teuthology.orchestra.run.smithi196.stdout:loop 2024-02-05T06:22:39.281 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-02-05T06:22:39.282 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-05T06:22:39.319 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-02-05T06:22:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_11 2024-02-05T06:22:39.385 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-02-05T06:22:39.386 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-05T06:22:39.465 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-02-05T06:22:39.504 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_21 2024-02-05T06:22:39.532 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-02-05T06:22:39.533 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-05T06:22:39.610 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-02-05T06:22:39.647 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_31 2024-02-05T06:22:39.683 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-02-05T06:22:39.683 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-05T06:22:39.761 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-02-05T06:22:39.798 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_41 2024-02-05T06:22:39.826 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:22:39.827 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:22:39.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme list 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/nvme0n1 CVFT6246000Z400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/nvme1n1 a2bb6e9c603247bfe352 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/nvme2n1 117b8ef191368d3b1105 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/nvme3n1 b973b9cc1e929148e77c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:39.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/nvme4n1 924adb21b2d0d54656e8 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-05T06:22:39.948 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:22:39.948 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:22:39.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-02-05T06:22:40.015 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-05T06:22:40.111 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921'} 2024-02-05T06:22:40.111 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:22:40.112 INFO:tasks.cephadm:Cluster fsid is f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:22:40.112 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-05T06:22:40.112 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.122', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.196'} 2024-02-05T06:22:40.112 INFO:tasks.cephadm:First mon is mon.a on smithi122 2024-02-05T06:22:40.112 INFO:tasks.cephadm:First mgr is y 2024-02-05T06:22:40.112 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-05T06:22:40.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-02-05T06:22:40.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-02-05T06:22:40.176 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-02-05T06:22:40.203 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-05T06:22:40.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:22:40.427 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76505/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1241-g164d3be4', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1241.g164d3be4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-05 00:32:51.050045', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-3-2024-02-04-2254', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-05T06:22:40.587 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-guits-testing-3-2024-02-04-2254, sha1 164d3be4a527be775f4d3c343ea591122f1b8921 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:22:40.588 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm 2024-02-05T06:22:40.588 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:41.004 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 766928 Feb 5 06:22 /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:41.005 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:41.602 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 766928 Feb 5 06:22 /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:41.602 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:42.188 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 766928 Feb 5 06:22 /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:42.189 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:42.208 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:42.227 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-05T06:22:42.258 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 on all hosts... 2024-02-05T06:22:42.258 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T06:22:42.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T06:22:42.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T06:22:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T06:22:43.094 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T06:22:43.159 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T06:23:13.921 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-02-05T06:23:13.922 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-05T06:23:29.623 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-02-05T06:23:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T06:23:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T06:23:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-02-05T06:23:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T06:23:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-02-05T06:23:29.624 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-02-05T06:23:36.437 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-02-05T06:23:36.462 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-02-05T06:23:36.495 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-02-05T06:23:36.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-02-05T06:23:36.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-02-05T06:23:36.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-02-05T06:23:36.618 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-02-05T06:23:36.645 INFO:tasks.cephadm:Writing seed config... 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-05T06:23:36.646 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-05T06:23:36.647 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-05T06:23:36.647 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:23:36.647 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-05T06:23:36.664 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f68cd702-c3ee-11ee-95b6-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-05T06:23:36.664 DEBUG:teuthology.orchestra.run.smithi122:mon.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service 2024-02-05T06:23:36.706 DEBUG:teuthology.orchestra.run.smithi122:mgr.y> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y.service 2024-02-05T06:23:36.749 INFO:tasks.cephadm:Bootstrapping... 2024-02-05T06:23:36.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 -v bootstrap --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.122 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:23:36.942 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-02-05T06:23:36.943 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921', '-v', 'bootstrap', '--fsid', 'f68cd702-c3ee-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.122', '--skip-admin-label'] 2024-02-05T06:23:36.967 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.0 2024-02-05T06:23:36.967 INFO:teuthology.orchestra.run.smithi122.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-05T06:23:36.968 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying podman|docker is present... 2024-02-05T06:23:36.992 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.0 2024-02-05T06:23:36.992 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying lvm2 is present... 2024-02-05T06:23:36.992 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying time synchronization is in place... 2024-02-05T06:23:36.999 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T06:23:36.999 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T06:23:37.005 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T06:23:37.005 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:23:37.011 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-02-05T06:23:37.017 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-02-05T06:23:37.017 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-02-05T06:23:37.017 INFO:teuthology.orchestra.run.smithi122.stdout:Repeating the final host check... 2024-02-05T06:23:37.041 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.0 2024-02-05T06:23:37.041 INFO:teuthology.orchestra.run.smithi122.stdout:podman (/bin/podman) version 4.9.0 is present 2024-02-05T06:23:37.042 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl is present 2024-02-05T06:23:37.042 INFO:teuthology.orchestra.run.smithi122.stdout:lvcreate is present 2024-02-05T06:23:37.048 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T06:23:37.048 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T06:23:37.053 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T06:23:37.053 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:23:37.060 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:Host looks OK 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:Cluster fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:Acquiring lock 139940443829824 on /run/cephadm/f68cd702-c3ee-11ee-95b6-87774f69a715.lock 2024-02-05T06:23:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:Lock 139940443829824 acquired on /run/cephadm/f68cd702-c3ee-11ee-95b6-87774f69a715.lock 2024-02-05T06:23:37.067 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 3300 ... 2024-02-05T06:23:37.067 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 6789 ... 2024-02-05T06:23:37.067 INFO:teuthology.orchestra.run.smithi122.stdout:Base mon IP(s) is [172.21.15.122:3300, 172.21.15.122:6789], mon addrv is [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-02-05T06:23:37.070 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.122 metric 100 2024-02-05T06:23:37.070 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.122 metric 100 2024-02-05T06:23:37.072 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-05T06:23:37.072 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-05T06:23:37.072 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1765sec hoplimit 64 pref medium 2024-02-05T06:23:37.074 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-02-05T06:23:37.075 INFO:teuthology.orchestra.run.smithi122.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-05T06:23:37.076 INFO:teuthology.orchestra.run.smithi122.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-05T06:23:37.076 INFO:teuthology.orchestra.run.smithi122.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T06:23:37.566 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Getting image source signatures 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:48ac04de0ee67de328b3ea21477b81735e6e7c0783ebcda067ca4695a8266750 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying config sha256:8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T06:23:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-05T06:23:42.728 INFO:teuthology.orchestra.run.smithi122.stdout:ceph: stdout ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T06:23:42.729 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph version: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T06:23:42.729 INFO:teuthology.orchestra.run.smithi122.stdout:Extracting ceph user uid/gid from container image... 2024-02-05T06:23:47.387 INFO:teuthology.orchestra.run.smithi122.stdout:stat: stdout 167 167 2024-02-05T06:23:47.387 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial keys... 2024-02-05T06:23:48.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQDzfsBl7fRTKxAA9+OaSnuqoAEuqLrbZYBtmw== 2024-02-05T06:23:49.398 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQD0fsBlhpq4KxAABhWxSdXjzyiJVawSPUE1BQ== 2024-02-05T06:23:50.711 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQD1fsBlABJ2LRAAxI9PHmLLdDkZXG8BoHZgPw== 2024-02-05T06:23:50.715 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial monmap... 2024-02-05T06:23:51.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T06:23:51.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-05T06:23:51.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:51.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T06:23:51.625 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool for a [v2:172.21.15.122:3300,v1:172.21.15.122:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T06:23:51.626 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-02-05T06:23:51.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: set fsid to f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:51.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T06:23:51.626 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:23:51.626 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mon... 2024-02-05T06:23:53.464 INFO:teuthology.orchestra.run.smithi122.stdout:create mon.a on 2024-02-05T06:23:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-05T06:23:54.031 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target. 2024-02-05T06:23:54.031 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target. 2024-02-05T06:23:54.280 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a 2024-02-05T06:23:54.280 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-05T06:23:54.436 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target.wants/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@.service. 2024-02-05T06:23:54.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 podman[24983]: 2024-02-05 06:23:54.581941568 +0000 UTC m=+0.017131036 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:23:54.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 podman[24983]: 2024-02-05 06:23:54.708381284 +0000 UTC m=+0.143570745 container create 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2) 2024-02-05T06:23:54.945 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T06:23:54.945 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-02-05T06:23:54.952 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T06:23:54.952 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:23:54.952 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-02-05T06:23:54.952 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T06:23:54.953 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon to start... 2024-02-05T06:23:54.953 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon... 2024-02-05T06:23:55.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 podman[24983]: 2024-02-05 06:23:54.877220415 +0000 UTC m=+0.312409874 container init 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-05T06:23:55.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 podman[24983]: 2024-02-05 06:23:54.882474542 +0000 UTC m=+0.317664003 container start 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:23:55.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 bash[24983]: 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 2024-02-05T06:23:55.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 systemd[1]: Started Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:23:55.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 ceph-mon[25021]: mkfs f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:55.075 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:54 smithi122 ceph-mon[25021]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: monmap e1: 1 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: fsmap 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: mgrmap e1: no daemons active 2024-02-05T06:23:56.013 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:55 smithi122 ceph-mon[25021]: from='client.? 172.21.15.122:0/2417320970' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-05T06:23:56.017 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout cluster: 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout id: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout services: 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.520807s) 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout data: 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-05T06:23:56.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-05T06:23:56.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pgs: 2024-02-05T06:23:56.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:56.019 INFO:teuthology.orchestra.run.smithi122.stdout:mon is available 2024-02-05T06:23:56.019 INFO:teuthology.orchestra.run.smithi122.stdout:Assimilating anything we can from ceph.conf... 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T06:23:57.082 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T06:23:57.083 INFO:teuthology.orchestra.run.smithi122.stdout:Generating new minimal ceph.conf... 2024-02-05T06:23:57.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:56 smithi122 ceph-mon[25021]: from='client.? 172.21.15.122:0/2046834849' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T06:23:57.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:56 smithi122 ceph-mon[25021]: from='client.? 172.21.15.122:0/2046834849' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-05T06:23:58.324 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:57 smithi122 ceph-mon[25021]: from='client.? 172.21.15.122:0/4044306041' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:23:58.621 INFO:teuthology.orchestra.run.smithi122.stdout:Restarting the monitor... 2024-02-05T06:23:58.873 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:58 smithi122 systemd[1]: Stopping Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:23:59.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:58 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a[24998]: 2024-02-05T06:23:58.883+0000 7f018d9b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:23:59.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:58 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a[24998]: 2024-02-05T06:23:58.883+0000 7f018d9b9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-05T06:23:59.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:58 smithi122 podman[25356]: 2024-02-05 06:23:58.956261329 +0000 UTC m=+0.157993736 container died 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-05T06:23:59.512 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 podman[25356]: 2024-02-05 06:23:59.323087552 +0000 UTC m=+0.524819959 container cleanup 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-05T06:23:59.513 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 bash[25356]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a 2024-02-05T06:23:59.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 podman[25375]: 2024-02-05 06:23:59.524197157 +0000 UTC m=+0.567297517 container remove 4cccb433622f85b0c6760008336f0da7d21366893dd5c50f0a3f959ea15313d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2) 2024-02-05T06:23:59.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-05T06:23:59.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 systemd[1]: Stopped Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:23:59.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 systemd[1]: Starting Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:24:00.086 INFO:teuthology.orchestra.run.smithi122.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-05T06:24:00.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 podman[25474]: 2024-02-05 06:23:59.866721999 +0000 UTC m=+0.114840673 container create b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:24:00.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:23:59 smithi122 podman[25474]: 2024-02-05 06:23:59.76947103 +0000 UTC m=+0.017589706 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 podman[25474]: 2024-02-05 06:24:00.028836295 +0000 UTC m=+0.276954976 container init b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 podman[25474]: 2024-02-05 06:24:00.034331148 +0000 UTC m=+0.282449821 container start b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: pidfile_write: ignore empty --pid-file 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: load: jerasure load: lrc 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: RocksDB version: 7.9.2 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Git sha 0 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: DB SUMMARY 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: DB Session ID: 98VZ6JPMG6SZROXSTLNH 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: CURRENT file: CURRENT 2024-02-05T06:24:00.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T06:24:00.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-05T06:24:00.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-05T06:24:00.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84657 ; 2024-02-05T06:24:00.124 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.error_if_exists: 0 2024-02-05T06:24:00.124 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.create_if_missing: 0 2024-02-05T06:24:00.124 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.paranoid_checks: 1 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.env: 0x55708b3a8ae0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.info_log: 0x55708c4f78c0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.statistics: (nil) 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.use_fsync: 0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_log_file_size: 0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T06:24:00.125 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_fallocate: 1 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.use_direct_reads: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.db_log_dir: 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.wal_dir: 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.write_buffer_manager: 0x55708c4cb400 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.unordered_write: 0 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.row_cache: None 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.wal_filter: None 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.two_write_queues: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.wal_compression: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.atomic_flush: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.log_readahead_size: 0 2024-02-05T06:24:00.128 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_background_jobs: 2 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_background_compactions: -1 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_subcompactions: 1 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T06:24:00.129 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_open_files: -1 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_background_flushes: -1 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Compression algorithms supported: 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kZSTD supported: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kXpressCompression supported: 0 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kZlibCompression supported: 1 2024-02-05T06:24:00.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kSnappyCompression supported: 1 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kLZ4Compression supported: 1 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: kBZip2Compression supported: 0 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.merge_operator: 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_filter: None 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55708c4f79c0) 2024-02-05T06:24:00.131 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_type: 0 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_index_type: 0 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_shortening: 1 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: checksum: 4 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: no_block_cache: 0 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache: 0x55708c4fb090 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_options: 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: capacity : 536870912 2024-02-05T06:24:00.132 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_shard_bits : 4 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: strict_capacity_limit : 0 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_compressed: (nil) 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: persistent_cache: (nil) 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size: 4096 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size_deviation: 10 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_restart_interval: 16 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_block_restart_interval: 1 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: metadata_block_size: 4096 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: partition_filters: 0 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: use_delta_encoding: 1 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: filter_policy: bloomfilter 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: whole_key_filtering: 1 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: verify_compression: 0 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-02-05T06:24:00.133 INFO:journalctl@ceph.mon.a.smithi122.stdout: format_version: 5 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_index_compression: 1 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_align: 0 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: max_auto_readahead_size: 262144 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: prepopulate_block_cache: 0 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: initial_auto_readahead_size: 8192 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression: NoCompression 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.num_levels: 7 2024-02-05T06:24:00.134 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T06:24:00.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T06:24:00.136 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T06:24:00.137 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.table_properties_collectors: 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.inplace_update_support: 0 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T06:24:00.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.bloom_locality: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.max_successive_merges: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.ttl: 2592000 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enable_blob_files: false 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.min_blob_size: 0 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T06:24:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2fef040-e5f4-461e-b423-ae1034a68252 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114240063103, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114240064012, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79469, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73778, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707114240, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2fef040-e5f4-461e-b423-ae1034a68252", "db_session_id": "98VZ6JPMG6SZROXSTLNH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114240064091, "job": 1, "event": "recovery_finished"} 2024-02-05T06:24:00.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55708c5ec000 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: DB pointer 0x55708c5d6000 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** DB Stats ** 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:24:00.141 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 81.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 81.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:24:00.142 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 26.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 26.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache BinnedLRUCache@0x55708c4fb090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: starting mon.a rank 0 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???) e1 preinit fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).mds e1 new map 2024-02-05T06:24:00.143 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).mds e1 print_map 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: e1 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: legacy client fscid: -1 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout: No filesystems configured 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 bash[25474]: b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 systemd[1]: Started Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:24:00.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T06:24:00.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: monmap e1: 1 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:24:00.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: fsmap 2024-02-05T06:24:00.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T06:24:00.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:00 smithi122 ceph-mon[25512]: mgrmap e1: no daemons active 2024-02-05T06:24:01.224 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-05T06:24:01.246 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:24:01.247 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mgr... 2024-02-05T06:24:01.247 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-05T06:24:01.247 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-05T06:24:01.490 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y 2024-02-05T06:24:01.490 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-05T06:24:01.641 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715.target.wants/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@.service. 2024-02-05T06:24:01.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:01 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/319429668' entity='client.admin' 2024-02-05T06:24:02.185 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:02 smithi122 podman[25752]: 2024-02-05 06:24:02.093082179 +0000 UTC m=+0.326004415 container start 9bd3bd6892db766e38730976234dc2bdba74ea03e9e4c0372147981724d9e5a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux ) 2024-02-05T06:24:02.185 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:02 smithi122 bash[25752]: 9bd3bd6892db766e38730976234dc2bdba74ea03e9e4c0372147981724d9e5a1 2024-02-05T06:24:02.198 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T06:24:02.199 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-02-05T06:24:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T06:24:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:24:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-02-05T06:24:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T06:24:02.215 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T06:24:02.215 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-02-05T06:24:02.222 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T06:24:02.222 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:24:02.223 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-02-05T06:24:02.223 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-05T06:24:02.223 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr to start... 2024-02-05T06:24:02.223 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr... 2024-02-05T06:24:02.467 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:02 smithi122 systemd[1]: Started Ceph mgr.y for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:24:02.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:02 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:02.647+0000 7f67ade0d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:24:02.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:02 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:02.744+0000 7f67ade0d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:24:03.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:02 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/4035498499' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T06:24:03.379 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f68cd702-c3ee-11ee-95b6-87774f69a715", 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T06:24:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:03.381 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T06:24:03.382 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T06:24:03.382 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T06:24:03.382 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T06:24:03.385 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T06:24:03.385 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T06:24:03.385 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.385 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T06:24:03.385 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T06:24:03.386 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T06:23:54.951727+0000", 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T06:24:03.387 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:03.388 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (1/15)... 2024-02-05T06:24:03.485 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:03 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:03.404+0000 7f67ade0d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:24:03.485 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:03 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:03.496+0000 7f67ade0d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:24:03.802 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:03 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:03.635+0000 7f67ade0d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:24:03.802 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:03 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:03.730+0000 7f67ade0d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:24:03.802 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:03 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:03.812+0000 7f67ade0d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:24:04.324 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:04 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:04.073+0000 7f67ade0d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:24:04.324 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:04 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:04.181+0000 7f67ade0d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:04.324 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:04 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:04.265+0000 7f67ade0d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:24:04.824 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:04 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:04.356+0000 7f67ade0d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:24:04.824 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:04 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:04.530+0000 7f67ade0d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:24:05.751 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:05 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:05.507+0000 7f67ade0d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:24:05.752 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:05 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:05.642+0000 7f67ade0d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:24:06.059 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:05 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:05.993+0000 7f67ade0d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:24:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1569316687' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T06:24:06.324 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:06.081+0000 7f67ade0d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:24:06.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:06.274+0000 7f67ade0d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:06.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f68cd702-c3ee-11ee-95b6-87774f69a715", 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T06:24:06.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T06:24:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T06:23:54.951727+0000", 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:06.551 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T06:24:06.551 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:06.551 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (2/15)... 2024-02-05T06:24:06.725 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:06.454+0000 7f67ade0d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:24:06.987 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:06.736+0000 7f67ade0d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:06.987 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:06 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:06.997+0000 7f67ade0d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:07.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:07 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:07.229+0000 7f67ade0d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:24:08.324 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:08.022+0000 7f67ade0d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:24:08.899 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:08.592+0000 7f67ade0d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:24:08.899 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:08.690+0000 7f67ade0d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:24:08.899 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:08.909+0000 7f67ade0d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: Activating manager daemon y 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: mgrmap e2: y(active, starting, since 0.00297593s) 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:24:09.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: Manager daemon y is now available 2024-02-05T06:24:09.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:24:09.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:24:09.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' 2024-02-05T06:24:09.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' 2024-02-05T06:24:09.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:08 smithi122 ceph-mon[25512]: from='mgr.14100 172.21.15.122:0/2569111143' entity='mgr.y' 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f68cd702-c3ee-11ee-95b6-87774f69a715", 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T06:24:09.636 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T06:24:09.637 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T06:24:09.638 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T06:23:54.951727+0000", 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:09.640 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (3/15)... 2024-02-05T06:24:10.324 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:09 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3406368315' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T06:24:10.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:09 smithi122 ceph-mon[25512]: mgrmap e3: y(active, since 1.00738s) 2024-02-05T06:24:12.324 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:12 smithi122 ceph-mon[25512]: mgrmap e4: y(active, since 2s) 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f68cd702-c3ee-11ee-95b6-87774f69a715", 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T06:24:12.898 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T06:24:12.899 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T06:24:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T06:24:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T06:24:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T06:24:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T06:24:12.903 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.903 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T06:24:12.903 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T06:24:12.903 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T06:24:12.904 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T06:24:12.905 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T06:23:54.951727+0000", 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-02-05T06:24:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T06:24:12.907 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:12.907 INFO:teuthology.orchestra.run.smithi122.stdout:mgr is available 2024-02-05T06:24:13.151 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:13 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1227110969' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T06:24:14.060 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T06:24:14.061 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling cephadm module... 2024-02-05T06:24:14.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:14 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/401310196' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T06:24:14.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:14 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/401310196' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-05T06:24:15.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:15 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1356512717' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-05T06:24:15.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setuser ceph since I am not root 2024-02-05T06:24:15.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setgroup ceph since I am not root 2024-02-05T06:24:15.881 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:15.633+0000 7fdc07d98200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:24:15.882 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:15.734+0000 7fdc07d98200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:24:16.324 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1356512717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-05T06:24:16.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-mon[25512]: mgrmap e5: y(active, since 6s) 2024-02-05T06:24:16.616 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:16.392+0000 7fdc07d98200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:24:16.616 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:16.485+0000 7fdc07d98200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:24:16.616 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:16.626+0000 7fdc07d98200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:24:16.946 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:16.724+0000 7fdc07d98200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:24:16.946 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:16 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:16.808+0000 7fdc07d98200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:24:16.949 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-02-05T06:24:16.950 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 5... 2024-02-05T06:24:17.208 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/854251358' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T06:24:17.209 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:17.075+0000 7fdc07d98200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:24:17.209 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:17.187+0000 7fdc07d98200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:17.541 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:17.279+0000 7fdc07d98200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:24:17.541 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:17.373+0000 7fdc07d98200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:24:17.824 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:17.551+0000 7fdc07d98200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:24:18.825 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:18.496+0000 7fdc07d98200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:24:18.825 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:18.623+0000 7fdc07d98200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:24:19.226 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:18.953+0000 7fdc07d98200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:24:19.226 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:19.043+0000 7fdc07d98200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:24:19.226 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:19.236+0000 7fdc07d98200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:19.575 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:19.419+0000 7fdc07d98200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:24:19.948 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:19.701+0000 7fdc07d98200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:19.948 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:19.958+0000 7fdc07d98200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:20.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:20 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:20.193+0000 7fdc07d98200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:24:21.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:20 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:20.986+0000 7fdc07d98200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:24:21.825 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:21.565+0000 7fdc07d98200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:24:21.825 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:21.661+0000 7fdc07d98200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:24:22.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:21.873+0000 7fdc07d98200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:24:22.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: Active manager daemon y restarted 2024-02-05T06:24:22.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: Activating manager daemon y 2024-02-05T06:24:22.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: osdmap e2: 0 total, 0 up, 0 in 2024-02-05T06:24:22.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: mgrmap e6: y(active, starting, since 0.00330089s) 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: Manager daemon y is now available 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:22.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:21 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: Found migration_current of "None". Setting to last migration. 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:23.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:22 smithi122 ceph-mon[25512]: mgrmap e7: y(active, since 1.00529s) 2024-02-05T06:24:23.480 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:23.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-05T06:24:23.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T06:24:23.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:23.481 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 5 is available 2024-02-05T06:24:23.481 INFO:teuthology.orchestra.run.smithi122.stdout:Setting orchestrator backend to cephadm... 2024-02-05T06:24:24.168 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:22] ENGINE Bus STARTING 2024-02-05T06:24:24.168 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:22] ENGINE Serving on http://172.21.15.122:8765 2024-02-05T06:24:24.168 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T06:24:24.168 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T06:24:24.169 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:22] ENGINE Serving on https://172.21.15.122:7150 2024-02-05T06:24:24.169 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:22] ENGINE Bus STARTED 2024-02-05T06:24:24.169 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:23 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:25.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:24 smithi122 ceph-mon[25512]: mgrmap e8: y(active, since 2s) 2024-02-05T06:24:25.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:24 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:25.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:24 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:25.908 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-05T06:24:25.909 INFO:teuthology.orchestra.run.smithi122.stdout:Generating ssh key... 2024-02-05T06:24:26.258 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:26 smithi122 ceph-mon[25512]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:27.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-mon[25512]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:27.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: Generating public/private rsa key pair. 2024-02-05T06:24:27.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: Your identification has been saved in /tmp/tmpua3w2ex5/key. 2024-02-05T06:24:27.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: Your public key has been saved in /tmp/tmpua3w2ex5/key.pub. 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: The key fingerprint is: 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: SHA256:jNVNYTksi7s93t1sHlhljnxc7ZliKadiWu1IY1TAzyM ceph-f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: The key's randomart image is: 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: +---[RSA 3072]----+ 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | .. .+o | 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | oo++ .| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | ..++.. =| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | +.Eo+ ..*=| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | . So...=o+=| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | o . = +. | 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | X o . . | 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | B B. . oo| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: | . o.o. .o+| 2024-02-05T06:24:27.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: +----[SHA256]-----+ 2024-02-05T06:24:28.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:28 smithi122 ceph-mon[25512]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:28.131 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:28 smithi122 ceph-mon[25512]: Generating ssh key... 2024-02-05T06:24:28.132 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:28 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:28.132 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:28 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:28.911 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:28.912 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-05T06:24:28.912 INFO:teuthology.orchestra.run.smithi122.stdout:Adding key to root@localhost authorized_keys... 2024-02-05T06:24:28.912 INFO:teuthology.orchestra.run.smithi122.stdout:Adding host smithi122... 2024-02-05T06:24:29.303 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:29 smithi122 ceph-mon[25512]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:31.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:31 smithi122 ceph-mon[25512]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:32.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:32 smithi122 ceph-mon[25512]: Deploying cephadm binary to smithi122 2024-02-05T06:24:34.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Added host 'smithi122' with addr '172.21.15.122' 2024-02-05T06:24:34.361 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mon service... 2024-02-05T06:24:35.023 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:34 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:35.023 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:34 smithi122 ceph-mon[25512]: Added host smithi122 2024-02-05T06:24:35.023 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:34 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:35.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-05T06:24:35.627 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mgr service... 2024-02-05T06:24:36.189 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:36 smithi122 ceph-mon[25512]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:36.189 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:36 smithi122 ceph-mon[25512]: Saving service mon spec with placement count:5 2024-02-05T06:24:36.189 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:36 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:37.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-05T06:24:38.153 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:37 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:38.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:37 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:38.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:37 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:38.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:38 smithi122 ceph-mon[25512]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:38.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:38 smithi122 ceph-mon[25512]: Saving service mgr spec with placement count:2 2024-02-05T06:24:38.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:38 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1644842713' entity='client.admin' 2024-02-05T06:24:40.548 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling the dashboard module... 2024-02-05T06:24:41.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:40 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3260269777' entity='client.admin' 2024-02-05T06:24:42.426 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-mon[25512]: from='mgr.14120 172.21.15.122:0/2208058183' entity='mgr.y' 2024-02-05T06:24:42.426 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1370897038' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-05T06:24:42.427 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setuser ceph since I am not root 2024-02-05T06:24:42.427 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setgroup ceph since I am not root 2024-02-05T06:24:42.756 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:42.672+0000 7f30c6789200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:24:42.757 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:42 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:42.766+0000 7f30c6789200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:24:43.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1370897038' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-05T06:24:43.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-mon[25512]: mgrmap e9: y(active, since 20s) 2024-02-05T06:24:43.759 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:43.430+0000 7f30c6789200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:24:43.759 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:43.524+0000 7f30c6789200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:24:43.759 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:43.671+0000 7f30c6789200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:24:43.759 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:43.768+0000 7f30c6789200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:24:44.075 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:43.853+0000 7f30c6789200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-02-05T06:24:44.086 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 9... 2024-02-05T06:24:44.339 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3120568263' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T06:24:44.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:44.121+0000 7f30c6789200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:24:44.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:44.233+0000 7f30c6789200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:44.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:44.318+0000 7f30c6789200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:24:44.590 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:44.418+0000 7f30c6789200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:24:45.075 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:44 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:44.599+0000 7f30c6789200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:24:45.825 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:45 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:45.552+0000 7f30c6789200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:24:45.826 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:45 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:45.679+0000 7f30c6789200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:24:46.281 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:46 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:46.008+0000 7f30c6789200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:24:46.281 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:46 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:46.097+0000 7f30c6789200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:24:46.281 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:46 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:46.290+0000 7f30c6789200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:46.575 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:46 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:46.474+0000 7f30c6789200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:24:47.003 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:46 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:46.756+0000 7f30c6789200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:24:47.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:47 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:47.012+0000 7f30c6789200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:24:47.325 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:47 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:47.246+0000 7f30c6789200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:24:48.326 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:48.043+0000 7f30c6789200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:24:48.911 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:48.608+0000 7f30c6789200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:24:48.911 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:48.705+0000 7f30c6789200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:24:48.911 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:24:48.920+0000 7f30c6789200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: Active manager daemon y restarted 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: Activating manager daemon y 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: osdmap e3: 0 total, 0 up, 0 in 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: mgrmap e10: y(active, starting, since 0.00314084s) 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:24:49.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:24:49.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:24:49.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: Manager daemon y is now available 2024-02-05T06:24:49.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:49.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:24:50.198 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:24:50.199 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:24:50.199 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:49 smithi122 ceph-mon[25512]: mgrmap e11: y(active, since 1.00569s) 2024-02-05T06:24:50.568 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-02-05T06:24:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-05T06:24:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T06:24:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-02-05T06:24:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 9 is available 2024-02-05T06:24:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:Generating a dashboard self-signed certificate... 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:49] ENGINE Bus STARTING 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:49] ENGINE Serving on http://172.21.15.122:8765 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:49] ENGINE Serving on https://172.21.15.122:7150 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: [05/Feb/2024:06:24:49] ENGINE Bus STARTED 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T06:24:51.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:51 smithi122 ceph-mon[25512]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T06:24:52.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:52 smithi122 ceph-mon[25512]: mgrmap e12: y(active, since 2s) 2024-02-05T06:24:52.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:24:52.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:24:52.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:24:52.125 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-05T06:24:52.125 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial admin user... 2024-02-05T06:24:53.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:53 smithi122 ceph-mon[25512]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:53.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:53 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:24:53.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:53 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:24:54.183 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:54 smithi122 ceph-mon[25512]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:24:54.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BYw1WofHFeONTiEbLDhyo.a2bVuNU/BaNe5iy4xl75MnK15T83Qj.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707114293, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-05T06:24:54.186 INFO:teuthology.orchestra.run.smithi122.stdout:Fetching dashboard port number... 2024-02-05T06:24:55.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 8443 2024-02-05T06:24:55.770 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T06:24:55.770 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-02-05T06:24:55.777 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T06:24:55.777 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-02-05T06:24:55.777 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-02-05T06:24:55.777 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph Dashboard is now available at: 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout: URL: https://smithi122.front.sepia.ceph.com:8443/ 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout: User: admin 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout: Password: vef61o2ple 2024-02-05T06:24:55.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:55.780 INFO:teuthology.orchestra.run.smithi122.stdout:Saving cluster configuration to /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config directory 2024-02-05T06:24:55.780 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling autotune for osd_memory_target 2024-02-05T06:24:56.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:56 smithi122 ceph-mon[25512]: mgrmap e13: y(active, since 6s) 2024-02-05T06:24:56.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:56 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3212165802' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout:Or, if you are only running a single cluster on this host: 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-05T06:24:58.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: ceph telemetry on 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout:For more information see: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:24:58.291 INFO:teuthology.orchestra.run.smithi122.stdout:Bootstrap complete. 2024-02-05T06:24:58.333 INFO:tasks.cephadm:Fetching config... 2024-02-05T06:24:58.334 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:24:58.334 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-05T06:24:58.350 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-05T06:24:58.350 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:24:58.350 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-05T06:24:58.406 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-05T06:24:58.406 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:24:58.406 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-05T06:24:58.474 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-05T06:24:58.474 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:24:58.474 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-05T06:24:58.528 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-05T06:24:58.529 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T06:24:58.614 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:58.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T06:24:58.673 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:58.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T06:24:58.734 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIijVxtoqF3+OOIWl/MJxWWjQazldnFDSpYhiGs3nZryWyHtWq8zfXjCgfibI5VgU2PKUInS2LzpjJuEobmZQzwDtVQORzedYArv92E0+b4xOt3Z7Kl+I+sStvKrWEo5XTJBl8sXqxFBQ8AmzuiVCwfzJenHIH5v/s2212cqFVbv3ra+4cTf1VGI9C0asJNEAJSZvsqTXfF38JRQ3QWYQCP+2GEbU4mvgqyNrF9Cw9dsCwVV01z51B8h+rJr1kgTpwrsfH1M1IMO75OQ3xu7L2Y5gensOKAc7D8dPUbq9ypwPFwiwCQJcdgF9blJGBAXwd+siqN8as5Zn96ABHxLpDCS49LG28n8VQPxL/xn+tKLIbo0lKGewqoht+LXvxubASe2TKGYULzpSehtqxxIwSpQx6zZoF606zrQ2tWbMqsohD3gWYor5qxQkBnG8wJsD0KWcd+nTwbvHzii6Zj0s7Pq/HapT/8Bv8O5RMY6oJ3S2TDuWEn1c7NKc/+G1Gtas= ceph-f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:24:58.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-05T06:24:58.986 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:24:59.076 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:24:58 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/2122844981' entity='client.admin' 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:01.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:01 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:02.016 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-05T06:25:02.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-05T06:25:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:02.624 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:02 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/2209981543' entity='client.admin' 2024-02-05T06:25:02.624 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:02.624 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:04.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-02-05T06:25:04.667 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:25:04.667 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-02-05T06:25:04.685 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:25:04.685 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:04.743 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-02-05T06:25:04.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch host add smithi142 2024-02-05T06:25:04.985 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:05.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:05.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: Updating smithi122:/etc/ceph/ceph.conf 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:05.206 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:06 smithi122 ceph-mon[25512]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:06 smithi122 ceph-mon[25512]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:06 smithi122 ceph-mon[25512]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:07.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:07 smithi122 ceph-mon[25512]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:08.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:08 smithi122 ceph-mon[25512]: Deploying cephadm binary to smithi142 2024-02-05T06:25:09.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:09 smithi122 ceph-mon[25512]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:11.037 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-02-05T06:25:11.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-05T06:25:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:12.245 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:12 smithi122 ceph-mon[25512]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:12.245 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:12 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:12.245 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:12 smithi122 ceph-mon[25512]: Added host smithi142 2024-02-05T06:25:12.245 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:12 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:12.246 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:12 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:13.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:13 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:13.574 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:25:13.574 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-02-05T06:25:14.164 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-02-05T06:25:14.165 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:25:14.165 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-02-05T06:25:14.183 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:25:14.183 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:14.240 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-02-05T06:25:14.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch host add smithi196 2024-02-05T06:25:14.473 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:14.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:14 smithi122 ceph-mon[25512]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:14.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:14 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:15.268 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:15 smithi122 ceph-mon[25512]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:16.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:16 smithi122 ceph-mon[25512]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:17.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:17.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:17.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:17.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:17.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:17.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:17.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:17.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:18.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:25:18.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:18.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: Deploying cephadm binary to smithi196 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:18.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:18 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:20.554 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-02-05T06:25:20.554 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:20 smithi122 ceph-mon[25512]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:21.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-05T06:25:21.387 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:21.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:21 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:21.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:21 smithi122 ceph-mon[25512]: Added host smithi196 2024-02-05T06:25:21.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:21 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:21.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:21 smithi122 ceph-mon[25512]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:21.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:21 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:22.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:22 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:23.186 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:25:23.187 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-02-05T06:25:23.744 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-05T06:25:23.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-05T06:25:23.973 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:24.439 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:24 smithi122 ceph-mon[25512]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:24.439 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:24 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:25.362 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:25 smithi122 ceph-mon[25512]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:26.155 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-02-05T06:25:26.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:26 smithi122 ceph-mon[25512]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:26.457 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:26 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T06:25:26.718 INFO:tasks.cephadm:Adding mon.a on smithi122 2024-02-05T06:25:26.718 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-02-05T06:25:26.718 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-02-05T06:25:26.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c' 2024-02-05T06:25:26.954 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:27.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:27.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:28.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:28.656 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-02-05T06:25:29.250 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.b.service 2024-02-05T06:25:29.253 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.c.service 2024-02-05T06:25:29.255 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T06:25:29.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c;count:3 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:29.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:29.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:29.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: Deploying daemon mon.c on smithi196 2024-02-05T06:25:29.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:29 smithi122 ceph-mon[25512]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:25:32.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:32 smithi122 ceph-mon[25512]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:32.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:32 smithi196 systemd[1]: Starting Ceph mon.c for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:25:32.998 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:32 smithi196 podman[30447]: 2024-02-05 06:25:32.756049258 +0000 UTC m=+0.197425663 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:25:32.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:32 smithi196 podman[30447]: 2024-02-05 06:25:32.8914547 +0000 UTC m=+0.332831097 container create cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True) 2024-02-05T06:25:33.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 podman[30447]: 2024-02-05 06:25:33.144289631 +0000 UTC m=+0.585666029 container init cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0) 2024-02-05T06:25:33.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 podman[30447]: 2024-02-05 06:25:33.149950697 +0000 UTC m=+0.591327094 container start cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-05T06:25:33.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pidfile_write: ignore empty --pid-file 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: load: jerasure load: lrc 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: RocksDB version: 7.9.2 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Git sha 0 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: DB SUMMARY 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: DB Session ID: NS0Y2GYRSGCH62UWCBDI 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: CURRENT file: CURRENT 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-05T06:25:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.error_if_exists: 0 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.create_if_missing: 0 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.paranoid_checks: 1 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.env: 0x5586af799ae0 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.info_log: 0x5586b1e998c0 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T06:25:33.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.statistics: (nil) 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.use_fsync: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_log_file_size: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_fallocate: 1 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.use_direct_reads: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.db_log_dir: 2024-02-05T06:25:33.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.wal_dir: 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.write_buffer_manager: 0x5586b1e6d400 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T06:25:33.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.unordered_write: 0 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.row_cache: None 2024-02-05T06:25:33.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.wal_filter: None 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.two_write_queues: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.wal_compression: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.atomic_flush: 0 2024-02-05T06:25:33.525 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.log_readahead_size: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_background_jobs: 2 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_background_compactions: -1 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_subcompactions: 1 2024-02-05T06:25:33.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_open_files: -1 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_background_flushes: -1 2024-02-05T06:25:33.527 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Compression algorithms supported: 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kZSTD supported: 0 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kXpressCompression supported: 0 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kZlibCompression supported: 1 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kSnappyCompression supported: 1 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kLZ4Compression supported: 1 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: kBZip2Compression supported: 0 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T06:25:33.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.merge_operator: 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_filter: None 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5586b1e999c0) 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T06:25:33.529 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x5586b1e9d090 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-02-05T06:25:33.530 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T06:25:33.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression: NoCompression 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.num_levels: 7 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T06:25:33.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T06:25:33.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T06:25:33.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T06:25:33.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.table_properties_collectors: 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.inplace_update_support: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.bloom_locality: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.max_successive_merges: 0 2024-02-05T06:25:33.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.ttl: 2592000 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enable_blob_files: false 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.min_blob_size: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T06:25:33.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a3222e35-a16a-43d6-979f-f9ba2e2fac35 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114333191156, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114333191694, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707114333, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a3222e35-a16a-43d6-979f-f9ba2e2fac35", "db_session_id": "NS0Y2GYRSGCH62UWCBDI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114333191776, "job": 1, "event": "recovery_finished"} 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5586b1f8e000 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: DB pointer 0x5586b1f78000 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T06:25:33.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:25:33.539 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T06:25:33.540 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x5586b1e9d090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(???) e0 preinit fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:33.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).mds e1 new map 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-05T06:25:33.542 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Deploying cephadm binary to smithi196 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Added host smithi196 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c;count:3 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: Deploying daemon mon.c on smithi196 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 ceph-mon[30516]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 bash[30447]: cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 2024-02-05T06:25:33.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:33 smithi196 systemd[1]: Started Ceph mon.c for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:25:38.243 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:25:38.243 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","modified":"2024-02-05T06:25:33.216340Z","created":"2024-02-05T06:23:51.090984Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-05T06:25:38.246 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-02-05T06:25:38.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:38.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: mon.a calling monitor election 2024-02-05T06:25:38.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='client.? 172.21.15.196:0/489925448' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:38.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: mon.c calling monitor election 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: monmap e2: 2 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: fsmap 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: mgrmap e13: y(active, since 49s) 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: overall HEALTH_OK 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:38.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:38.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: mon.a calling monitor election 2024-02-05T06:25:38.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='client.? 172.21.15.196:0/489925448' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:38.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: mon.c calling monitor election 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: monmap e2: 2 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: fsmap 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: mgrmap e13: y(active, since 49s) 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: overall HEALTH_OK 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:39.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:39.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:39.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:39.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: Deploying daemon mon.b on smithi142 2024-02-05T06:25:39.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:39.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:39 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: Deploying daemon mon.b on smithi142 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:39.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:39 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:39.848 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T06:25:39.848 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T06:25:40.093 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:25:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:25:41.972 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","modified":"2024-02-05T06:25:33.216340Z","created":"2024-02-05T06:23:51.090984Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-05T06:25:41.974 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-02-05T06:25:42.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 podman[30015]: 2024-02-05 06:25:41.956985315 +0000 UTC m=+0.017485644 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:25:42.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:42 smithi196 ceph-mon[30516]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:42 smithi196 ceph-mon[30516]: from='client.? 172.21.15.196:0/373061628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 podman[30015]: 2024-02-05 06:25:42.220510265 +0000 UTC m=+0.281010591 container init d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b, io.buildah.version=1.33.2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 podman[30015]: 2024-02-05 06:25:42.225567582 +0000 UTC m=+0.286067912 container start d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pidfile_write: ignore empty --pid-file 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: load: jerasure load: lrc 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: RocksDB version: 7.9.2 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Git sha 0 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T06:25:42.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: DB SUMMARY 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: DB Session ID: BB0Q1POOP7DWS2XB6KBI 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: CURRENT file: CURRENT 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.error_if_exists: 0 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.create_if_missing: 0 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.paranoid_checks: 1 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.env: 0x55f0d8cbfae0 2024-02-05T06:25:42.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T06:25:42.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.info_log: 0x55f0dab838c0 2024-02-05T06:25:42.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T06:25:42.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.statistics: (nil) 2024-02-05T06:25:42.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.use_fsync: 0 2024-02-05T06:25:42.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_log_file_size: 0 2024-02-05T06:25:42.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_fallocate: 1 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.use_direct_reads: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.db_log_dir: 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.wal_dir: 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T06:25:42.564 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.write_buffer_manager: 0x55f0dab57400 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T06:25:42.565 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.unordered_write: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.row_cache: None 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.wal_filter: None 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.two_write_queues: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.wal_compression: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.atomic_flush: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T06:25:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.log_readahead_size: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_background_jobs: 2 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_background_compactions: -1 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_subcompactions: 1 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T06:25:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_open_files: -1 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_background_flushes: -1 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Compression algorithms supported: 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T06:25:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kZSTD supported: 0 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kXpressCompression supported: 0 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kZlibCompression supported: 1 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kSnappyCompression supported: 1 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kLZ4Compression supported: 1 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: kBZip2Compression supported: 0 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.merge_operator: 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_filter: None 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T06:25:42.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f0dab839c0) 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x55f0dab87090 2024-02-05T06:25:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2024-02-05T06:25:42.571 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression: NoCompression 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T06:25:42.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.num_levels: 7 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T06:25:42.573 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T06:25:42.574 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T06:25:42.575 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T06:25:42.576 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.table_properties_collectors: 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.inplace_update_support: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.bloom_locality: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.max_successive_merges: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.ttl: 2592000 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T06:25:42.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enable_blob_files: false 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.min_blob_size: 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-05T06:25:42.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a27baad3-3545-45e0-901d-a5684953bb68 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114342254918, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114342255324, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707114342, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a27baad3-3545-45e0-901d-a5684953bb68", "db_session_id": "BB0Q1POOP7DWS2XB6KBI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707114342255401, "job": 1, "event": "recovery_finished"} 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f0dac78000 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: DB pointer 0x55f0dac62000 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:42.579 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.580 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T06:25:42.581 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x55f0dab87090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(???) e0 preinit fsid f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).mds e1 new map 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T06:25:42.582 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:42.584 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Deploying cephadm binary to smithi196 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Added host smithi196 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/501920349' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c;count:3 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Deploying daemon mon.c on smithi196 2024-02-05T06:25:42.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.a calling monitor election 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.? 172.21.15.196:0/489925448' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.c calling monitor election 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:42.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: monmap e2: 2 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: fsmap 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mgrmap e13: y(active, since 49s) 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: overall HEALTH_OK 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: Deploying daemon mon.b on smithi142 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: from='client.? 172.21.15.196:0/373061628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:42.590 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 ceph-mon[30053]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-05T06:25:42.590 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 bash[30015]: d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf 2024-02-05T06:25:42.590 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:42 smithi142 systemd[1]: Started Ceph mon.b for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:25:42.590 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:42 smithi122 ceph-mon[25512]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:42.590 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:42 smithi122 ceph-mon[25512]: from='client.? 172.21.15.196:0/373061628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T06:25:43.563 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T06:25:43.563 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T06:25:43.806 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:25:47.768 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: mon.a calling monitor election 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: mon.c calling monitor election 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:47.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: fsmap 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: mgrmap e13: y(active, since 58s) 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:47.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: mon.a calling monitor election 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: mon.c calling monitor election 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:47.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: fsmap 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: mgrmap e13: y(active, since 58s) 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:47.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:48.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-02-05T06:25:48.630 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","modified":"2024-02-05T06:25:42.277797Z","created":"2024-02-05T06:23:51.090984Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-05T06:25:48.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:48 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:48.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:48 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:48.633 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-02-05T06:25:48.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:48.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.500 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-05T06:25:49.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.a calling monitor election 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.c calling monitor election 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: fsmap 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mgrmap e13: y(active, since 58s) 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:49.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:49.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mon.b calling monitor election 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mon.b calling monitor election 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mon.c calling monitor election 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mon.a calling monitor election 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: fsmap 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: mgrmap e13: y(active, since 60s) 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-05T06:25:50.161 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: Cluster is now healthy 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: overall HEALTH_OK 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:50.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:50.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mon.b calling monitor election 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mon.b calling monitor election 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mon.c calling monitor election 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mon.a calling monitor election 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: fsmap 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: mgrmap e13: y(active, since 60s) 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: Cluster is now healthy 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: overall HEALTH_OK 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:50.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:50.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.b calling monitor election 2024-02-05T06:25:50.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.b calling monitor election 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.c calling monitor election 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.a calling monitor election 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: fsmap 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: mgrmap e13: y(active, since 60s) 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: Cluster is now healthy 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: overall HEALTH_OK 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:50.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:51.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:51 smithi122 ceph-mon[25512]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T06:25:51.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:51 smithi122 ceph-mon[25512]: Reconfiguring daemon mon.a on smithi122 2024-02-05T06:25:51.370 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:51.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:51 smithi196 ceph-mon[30516]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T06:25:51.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:51 smithi196 ceph-mon[30516]: Reconfiguring daemon mon.a on smithi122 2024-02-05T06:25:51.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:51.558 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:51 smithi142 ceph-mon[30053]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T06:25:51.558 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:51 smithi142 ceph-mon[30053]: Reconfiguring daemon mon.a on smithi122 2024-02-05T06:25:51.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:25:52.291 INFO:teuthology.orchestra.run.smithi122.stdout:# minimal ceph.conf for f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:52.291 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-02-05T06:25:52.291 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:25:52.291 INFO:teuthology.orchestra.run.smithi122.stdout: mon_host = [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-02-05T06:25:52.291 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:52.291 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.291 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.292 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:52.292 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:52.292 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:52.467 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:52.467 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.467 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.468 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:52.468 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:52.468 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:52.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:52.859 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-05T06:25:52.859 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:25:52.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T06:25:52.889 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:25:52.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:52.956 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:25:52.956 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T06:25:52.987 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:25:52.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:53.053 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:25:53.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T06:25:53.091 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:25:53.092 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:25:53.162 INFO:tasks.cephadm:Adding mgr.y on smithi122 2024-02-05T06:25:53.162 INFO:tasks.cephadm:Adding mgr.x on smithi142 2024-02-05T06:25:53.162 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-02-05T06:25:53.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply mgr '3;smithi122=y;smithi142=x;smithi196=z' 2024-02-05T06:25:53.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:53 smithi142 ceph-mon[30053]: Reconfiguring mon.b (monmap changed)... 2024-02-05T06:25:53.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:53 smithi142 ceph-mon[30053]: Reconfiguring daemon mon.b on smithi142 2024-02-05T06:25:53.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:53 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3728517715' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:53.439 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:25:53.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:53 smithi196 ceph-mon[30516]: Reconfiguring mon.b (monmap changed)... 2024-02-05T06:25:53.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:53 smithi196 ceph-mon[30516]: Reconfiguring daemon mon.b on smithi142 2024-02-05T06:25:53.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:53 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3728517715' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:53.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:53 smithi122 ceph-mon[25512]: Reconfiguring mon.b (monmap changed)... 2024-02-05T06:25:53.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:53 smithi122 ceph-mon[25512]: Reconfiguring daemon mon.b on smithi142 2024-02-05T06:25:53.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:53 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3728517715' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:54.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:54.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:54.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:54.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:54.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:54.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:54.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:54.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T06:25:54.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T06:25:54.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:55.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:55 smithi196 ceph-mon[30516]: Reconfiguring mon.c (monmap changed)... 2024-02-05T06:25:55.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:55 smithi196 ceph-mon[30516]: Reconfiguring daemon mon.c on smithi196 2024-02-05T06:25:55.558 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:55 smithi142 ceph-mon[30053]: Reconfiguring mon.c (monmap changed)... 2024-02-05T06:25:55.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:55 smithi142 ceph-mon[30053]: Reconfiguring daemon mon.c on smithi196 2024-02-05T06:25:55.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:55 smithi122 ceph-mon[25512]: Reconfiguring mon.c (monmap changed)... 2024-02-05T06:25:55.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:55 smithi122 ceph-mon[25512]: Reconfiguring daemon mon.c on smithi196 2024-02-05T06:25:56.170 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-02-05T06:25:56.496 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:56 smithi196 ceph-mon[30516]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:56.496 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:56 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.496 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:56 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.496 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:56 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:56.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:56 smithi142 ceph-mon[30053]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:56.615 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:56 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:56 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:56 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:56.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:56 smithi122 ceph-mon[25512]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:56.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:56 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:56 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:56.617 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:56 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:25:56.874 DEBUG:teuthology.orchestra.run.smithi142:mgr.x> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x.service 2024-02-05T06:25:56.876 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.z.service 2024-02-05T06:25:56.878 INFO:tasks.cephadm:Deploying OSDs... 2024-02-05T06:25:56.878 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:25:56.879 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:25:56.895 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:25:56.895 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/nvme1n1 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/nvme1n1 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:24:59.217116936 +0000 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:36.156496377 +0000 2024-02-05T06:25:56.950 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:36.156496377 +0000 2024-02-05T06:25:56.951 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:36.147496581 +0000 2024-02-05T06:25:56.951 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-05T06:25:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:25:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:25:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000213837 s, 2.4 MB/s 2024-02-05T06:25:57.018 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-05T06:25:57.074 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/nvme2n1 2024-02-05T06:25:57.129 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/nvme2n1 2024-02-05T06:25:57.129 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:57.129 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-05T06:25:57.129 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:57.129 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:57.130 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:24:59.231116618 +0000 2024-02-05T06:25:57.130 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:36.307492948 +0000 2024-02-05T06:25:57.130 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:36.307492948 +0000 2024-02-05T06:25:57.130 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:36.295493220 +0000 2024-02-05T06:25:57.130 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-05T06:25:57.194 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:25:57.194 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:25:57.195 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000230734 s, 2.2 MB/s 2024-02-05T06:25:57.196 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-05T06:25:57.251 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/nvme3n1 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/nvme3n1 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:24:59.242116368 +0000 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:36.456489565 +0000 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:36.456489565 +0000 2024-02-05T06:25:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:36.451489678 +0000 2024-02-05T06:25:57.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-05T06:25:57.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=y;smithi142=x;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: Saving service mgr spec with placement smithi122=y;smithi142=x;smithi196=z;count:3 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:25:57.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:57 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=y;smithi142=x;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:57.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: Saving service mgr spec with placement smithi122=y;smithi142=x;smithi196=z;count:3 2024-02-05T06:25:57.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:25:57.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:57 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:25:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:25:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00029898 s, 1.7 MB/s 2024-02-05T06:25:57.354 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-05T06:25:57.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=y;smithi142=x;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:25:57.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: Saving service mgr spec with placement smithi122=y;smithi142=x;smithi196=z;count:3 2024-02-05T06:25:57.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:25:57.391 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:25:57.391 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:25:57.391 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:25:57.391 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:25:57.391 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:57 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:25:57.410 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/nvme4n1 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/nvme4n1 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-02-05 06:24:59.254116095 +0000 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-02-05 06:22:36.606486158 +0000 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-02-05 06:22:36.606486158 +0000 2024-02-05T06:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-02-05 06:22:36.595486408 +0000 2024-02-05T06:25:57.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-05T06:25:57.530 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-02-05T06:25:57.531 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-02-05T06:25:57.531 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000235405 s, 2.2 MB/s 2024-02-05T06:25:57.532 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-05T06:25:57.588 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:25:57.588 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:25:57.607 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:25:57.607 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/nvme1n1 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/nvme1n1 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:57.665 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:25:15.291108837 +0000 2024-02-05T06:25:57.666 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:37.789110560 +0000 2024-02-05T06:25:57.666 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:37.789110560 +0000 2024-02-05T06:25:57.666 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:37.777113284 +0000 2024-02-05T06:25:57.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-05T06:25:57.734 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:25:57.734 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:25:57.734 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000303876 s, 1.7 MB/s 2024-02-05T06:25:57.735 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-05T06:25:57.792 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/nvme2n1 2024-02-05T06:25:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/nvme2n1 2024-02-05T06:25:57.849 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:25:15.302108632 +0000 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:37.934077637 +0000 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:37.934077637 +0000 2024-02-05T06:25:57.850 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:37.923080134 +0000 2024-02-05T06:25:57.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-05T06:25:57.916 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:25:57.917 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:25:57.917 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00023417 s, 2.2 MB/s 2024-02-05T06:25:57.918 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-05T06:25:57.975 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/nvme3n1 2024-02-05T06:25:58.030 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/nvme3n1 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 788 Links: 1 Device type: 103,6 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:25:15.312108445 +0000 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:38.083043805 +0000 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:38.083043805 +0000 2024-02-05T06:25:58.031 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:38.073046076 +0000 2024-02-05T06:25:58.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-05T06:25:58.097 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:25:58.097 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:25:58.098 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000220489 s, 2.3 MB/s 2024-02-05T06:25:58.099 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-05T06:25:58.154 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/nvme4n1 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/nvme4n1 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-05 06:25:15.323108239 +0000 2024-02-05T06:25:58.210 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-05 06:22:38.228036490 +0000 2024-02-05T06:25:58.211 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-05 06:22:38.228036490 +0000 2024-02-05T06:25:58.211 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-02-05 06:22:38.217036692 +0000 2024-02-05T06:25:58.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-05T06:25:58.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:58 smithi196 ceph-mon[30516]: Deploying daemon mgr.z on smithi196 2024-02-05T06:25:58.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:25:58 smithi196 ceph-mon[30516]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:58.275 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-05T06:25:58.276 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-05T06:25:58.276 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000218724 s, 2.3 MB/s 2024-02-05T06:25:58.277 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-05T06:25:58.332 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:25:58.333 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T06:25:58.348 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-05T06:25:58.348 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme1n1 2024-02-05T06:25:58.402 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme1n1 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:25:25.332474594 +0000 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:39.410004132 +0000 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:39.410004132 +0000 2024-02-05T06:25:58.403 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:39.399004366 +0000 2024-02-05T06:25:58.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-05T06:25:58.472 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:25:58.472 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:25:58.472 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000278327 s, 1.8 MB/s 2024-02-05T06:25:58.473 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-05T06:25:58.530 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme2n1 2024-02-05T06:25:58.558 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:58 smithi142 ceph-mon[30053]: Deploying daemon mgr.z on smithi196 2024-02-05T06:25:58.558 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:25:58 smithi142 ceph-mon[30053]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:58.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:58 smithi122 ceph-mon[25512]: Deploying daemon mgr.z on smithi196 2024-02-05T06:25:58.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:25:58 smithi122 ceph-mon[25512]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme2n1 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:25:25.343474359 +0000 2024-02-05T06:25:58.585 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:39.557001016 +0000 2024-02-05T06:25:58.586 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:39.557001016 +0000 2024-02-05T06:25:58.586 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:39.545001270 +0000 2024-02-05T06:25:58.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-05T06:25:58.651 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:25:58.651 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:25:58.651 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000229043 s, 2.2 MB/s 2024-02-05T06:25:58.652 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-05T06:25:58.707 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme3n1 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme3n1 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 788 Links: 1 Device type: 103,6 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:25:25.354474124 +0000 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:39.704997877 +0000 2024-02-05T06:25:58.762 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:39.704997877 +0000 2024-02-05T06:25:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:39.696998047 +0000 2024-02-05T06:25:58.763 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-05T06:25:58.831 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:25:58.831 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:25:58.831 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000216147 s, 2.4 MB/s 2024-02-05T06:25:58.832 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-05T06:25:58.888 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme4n1 2024-02-05T06:25:58.943 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme4n1 2024-02-05T06:25:58.943 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-05T06:25:58.943 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 798 Links: 1 Device type: 103,8 2024-02-05T06:25:58.943 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-05T06:25:58.943 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-05T06:25:58.944 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-02-05 06:25:25.364473911 +0000 2024-02-05T06:25:58.944 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-02-05 06:22:39.849994803 +0000 2024-02-05T06:25:58.944 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-02-05 06:22:39.849994803 +0000 2024-02-05T06:25:58.944 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-02-05 06:22:39.839995015 +0000 2024-02-05T06:25:58.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-05T06:25:59.011 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-02-05T06:25:59.012 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-02-05T06:25:59.012 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000239561 s, 2.1 MB/s 2024-02-05T06:25:59.013 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-05T06:25:59.070 INFO:tasks.cephadm:Deploying osd.0 on smithi122 with /dev/nvme4n1... 2024-02-05T06:25:59.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-05T06:25:59.255 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:25:59 smithi196 systemd[1]: Started Ceph mgr.z for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:25:59.330 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:25:59.768 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:25:59 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:25:59.519+0000 7f9a87f7b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:25:59.768 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:25:59 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:25:59.615+0000 7f9a87f7b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:00.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:00 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:00.451 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:00 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:00.503 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.279+0000 7f9a87f7b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.370+0000 7f9a87f7b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.510+0000 7f9a87f7b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:00.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:00.768 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.605+0000 7f9a87f7b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:26:00.768 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.686+0000 7f9a87f7b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:26:01.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:01 smithi196 ceph-mon[30516]: Deploying daemon mgr.x on smithi142 2024-02-05T06:26:01.225 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:00 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:00.946+0000 7f9a87f7b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:26:01.225 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:01 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:01.053+0000 7f9a87f7b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:26:01.225 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:01 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:01.137+0000 7f9a87f7b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:26:01.225 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:01 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:01.232+0000 7f9a87f7b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/nvme4n1 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120768 s, 868 MB/s 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 2024-02-05T06:26:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-02-05T06:26:01.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:01 smithi142 ceph-mon[30053]: Deploying daemon mgr.x on smithi142 2024-02-05T06:26:01.430 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 podman[31493]: 2024-02-05 06:26:01.24532771 +0000 UTC m=+0.017684034 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:26:01.518 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:01 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:01.409+0000 7f9a87f7b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:26:01.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:01 smithi122 ceph-mon[25512]: Deploying daemon mgr.x on smithi142 2024-02-05T06:26:01.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 podman[31493]: 2024-02-05 06:26:01.546802934 +0000 UTC m=+0.319159253 container init 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0) 2024-02-05T06:26:01.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 podman[31493]: 2024-02-05 06:26:01.552040439 +0000 UTC m=+0.324396762 container start 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131) 2024-02-05T06:26:01.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 ceph-mgr[31530]: -- 172.21.15.142:0/381313790 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a4f3d55520 con 0x55a4f3d2ac00 2024-02-05T06:26:01.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 bash[31493]: 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 2024-02-05T06:26:01.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:01 smithi142 systemd[1]: Started Ceph mgr.x for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:26:01.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi122:/dev/nvme4n1 2024-02-05T06:26:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.408 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.469 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:02.109+0000 7fc2385c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:02.206+0000 7fc2385c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:02.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:02.712 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:02.415+0000 7f9a87f7b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:26:02.712 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:02.559+0000 7f9a87f7b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:26:02.961 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:02.892+0000 7fc2385c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:26:02.988 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:02.905+0000 7f9a87f7b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:26:02.988 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:02 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:02.995+0000 7f9a87f7b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:26:03.234 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:02 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:02.985+0000 7fc2385c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:26:03.234 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.139+0000 7fc2385c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:26:03.234 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.241+0000 7fc2385c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:26:03.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:03 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:03.205+0000 7f9a87f7b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:26:03.559 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.330+0000 7fc2385c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:26:03.665 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:03 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:03.391+0000 7f9a87f7b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:26:03.890 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.606+0000 7fc2385c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:26:03.890 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.715+0000 7fc2385c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:26:03.890 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.802+0000 7fc2385c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:26:03.891 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:03 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:03.897+0000 7fc2385c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:26:03.927 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:03 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:03.672+0000 7f9a87f7b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:26:03.927 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:03 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:03.934+0000 7f9a87f7b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:26:04.145 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:04.077+0000 7fc2385c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:26:04.145 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:04.146 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.147 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:04 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:04.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:04 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.269 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:04.169+0000 7f9a87f7b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:26:04.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:04.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:04.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:04.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:04.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.268 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:04 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:04.968+0000 7f9a87f7b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: Reconfiguring daemon mgr.y on smithi122 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:05.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:05.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:05.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.055+0000 7fc2385c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.186+0000 7fc2385c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: Reconfiguring daemon mgr.y on smithi122 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:05.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:05.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:05.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: Reconfiguring daemon mgr.y on smithi122 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:05.393 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:05.393 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:05.393 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:05 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:05.798 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.521+0000 7fc2385c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:26:05.798 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.611+0000 7fc2385c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:26:05.798 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.805+0000 7fc2385c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:26:05.846 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:05.541+0000 7f9a87f7b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:26:05.847 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:05.638+0000 7f9a87f7b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:26:05.847 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:26:05 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:26:05.853+0000 7f9a87f7b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:26:06.059 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:05 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:05.989+0000 7fc2385c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: Standby manager daemon z started 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:26:06.388 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:06 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:06.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:06.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: Standby manager daemon z started 2024-02-05T06:26:06.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:26:06.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:06.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:26:06.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:06 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: Standby manager daemon z started 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/1375807619' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:06.524 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:06.270+0000 7fc2385c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:26:06.525 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:06.531+0000 7fc2385c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:26:06.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:06 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:06.765+0000 7fc2385c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:26:07.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:07 smithi196 ceph-mon[30516]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-05T06:26:07.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:07 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:26:07.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:07 smithi142 ceph-mon[30053]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-05T06:26:07.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:07 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:26:07.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:07 smithi122 ceph-mon[25512]: mgrmap e14: y(active, since 77s), standbys: z 2024-02-05T06:26:07.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:07 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:26:07.809 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:07 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:07.561+0000 7fc2385c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:26:08.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-mon[30053]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:08.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]: dispatch 2024-02-05T06:26:08.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]': finished 2024-02-05T06:26:08.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-mon[30053]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T06:26:08.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:08.438 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:08.132+0000 7fc2385c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:26:08.438 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:08.230+0000 7fc2385c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:26:08.438 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:26:08 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:26:08.444+0000 7fc2385c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:26:08.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:08 smithi196 ceph-mon[30516]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:08.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:08 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]: dispatch 2024-02-05T06:26:08.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:08 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]': finished 2024-02-05T06:26:08.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:08 smithi196 ceph-mon[30516]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T06:26:08.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:08 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:08.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:08 smithi122 ceph-mon[25512]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:08.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:08 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]: dispatch 2024-02-05T06:26:08.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:08 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/36497747' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda59134-6703-45f2-af25-a75ef3696bdb"}]': finished 2024-02-05T06:26:08.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:08 smithi122 ceph-mon[25512]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T06:26:08.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:08 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: Standby manager daemon x started 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:09.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:09 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/118207252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: Standby manager daemon x started 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:09.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:09 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/118207252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: Standby manager daemon x started 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2850763181' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:26:09.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:09 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/118207252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:10.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:10 smithi196 ceph-mon[30516]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:10.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:10 smithi196 ceph-mon[30516]: mgrmap e15: y(active, since 80s), standbys: z, x 2024-02-05T06:26:10.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:26:10.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:10 smithi142 ceph-mon[30053]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:10.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:10 smithi142 ceph-mon[30053]: mgrmap e15: y(active, since 80s), standbys: z, x 2024-02-05T06:26:10.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:26:10.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:10 smithi122 ceph-mon[25512]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:10.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:10 smithi122 ceph-mon[25512]: mgrmap e15: y(active, since 80s), standbys: z, x 2024-02-05T06:26:10.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:26:12.226 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:12 smithi122 ceph-mon[25512]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:12.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:12 smithi196 ceph-mon[30516]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:12.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:12 smithi142 ceph-mon[30053]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:14.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:14 smithi196 ceph-mon[30516]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:14.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:14 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T06:26:14.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:14 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:14.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:14 smithi142 ceph-mon[30053]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:14.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:14 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T06:26:14.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:14 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:14.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:14 smithi122 ceph-mon[25512]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:14.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:14 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T06:26:14.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:14 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:15.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:15 smithi122 ceph-mon[25512]: Deploying daemon osd.0 on smithi122 2024-02-05T06:26:15.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:15 smithi196 ceph-mon[30516]: Deploying daemon osd.0 on smithi122 2024-02-05T06:26:15.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:15 smithi142 ceph-mon[30053]: Deploying daemon osd.0 on smithi122 2024-02-05T06:26:16.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:16 smithi196 ceph-mon[30516]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:16.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:16 smithi142 ceph-mon[30053]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:16.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:16 smithi122 ceph-mon[25512]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:18.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:18 smithi196 ceph-mon[30516]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:18.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:18 smithi142 ceph-mon[30053]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:18.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:18 smithi122 ceph-mon[25512]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:20.416 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:20 smithi122 ceph-mon[25512]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:20.416 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.416 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.416 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:20.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:20 smithi196 ceph-mon[30516]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:20.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:20.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:20 smithi142 ceph-mon[30053]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:20.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:20.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:22.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:22 smithi196 ceph-mon[30516]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:22.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:22 smithi142 ceph-mon[30053]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:22.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:22 smithi122 ceph-mon[25512]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:23.366 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 0 on host 'smithi122' 2024-02-05T06:26:23.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:23.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T06:26:23.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:23.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:23.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:23.810 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:23.939 DEBUG:teuthology.orchestra.run.smithi122:osd.0> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.0.service 2024-02-05T06:26:23.942 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/nvme4n1... 2024-02-05T06:26:23.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-05T06:26:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T06:26:24.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:24 smithi142 ceph-mon[30053]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T06:26:24.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:24 smithi142 ceph-mon[30053]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T06:26:24.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:24 smithi142 ceph-mon[30053]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-02-05T06:26:24.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:24 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:24.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-mon[25512]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T06:26:24.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-mon[25512]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T06:26:24.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-mon[25512]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-02-05T06:26:24.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:24.579 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0[40537]: 2024-02-05T06:26:24.391+0000 7f8d905e1700 -1 osd.0 0 waiting for initial osdmap 2024-02-05T06:26:24.579 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:26:24 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0[40537]: 2024-02-05T06:26:24.394+0000 7f8d8c3db700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T06:26:24.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:24 smithi196 ceph-mon[30516]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T06:26:24.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:24 smithi196 ceph-mon[30516]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T06:26:24.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:24 smithi196 ceph-mon[30516]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-02-05T06:26:24.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:24 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.508 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:25 smithi142 ceph-mon[30053]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-02-05T06:26:25.508 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:25 smithi142 ceph-mon[30053]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T06:26:25.508 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.508 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.508 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:25 smithi142 ceph-mon[30053]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:25.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:25 smithi196 ceph-mon[30516]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-02-05T06:26:25.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:25 smithi196 ceph-mon[30516]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T06:26:25.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:25 smithi196 ceph-mon[30516]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:25.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:25 smithi122 ceph-mon[25512]: from='osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-02-05T06:26:25.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:25 smithi122 ceph-mon[25512]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T06:26:25.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:25.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:25 smithi122 ceph-mon[25512]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/nvme4n1 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124194 s, 844 MB/s 2024-02-05T06:26:26.077 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-02-05T06:26:26.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:26 smithi142 ceph-mon[30053]: purged_snaps scrub starts 2024-02-05T06:26:26.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:26 smithi142 ceph-mon[30053]: purged_snaps scrub ok 2024-02-05T06:26:26.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:26 smithi142 ceph-mon[30053]: osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912] boot 2024-02-05T06:26:26.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:26 smithi142 ceph-mon[30053]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T06:26:26.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:26 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:26.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi142:/dev/nvme4n1 2024-02-05T06:26:26.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:26 smithi122 ceph-mon[25512]: purged_snaps scrub starts 2024-02-05T06:26:26.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:26 smithi122 ceph-mon[25512]: purged_snaps scrub ok 2024-02-05T06:26:26.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:26 smithi122 ceph-mon[25512]: osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912] boot 2024-02-05T06:26:26.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:26 smithi122 ceph-mon[25512]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T06:26:26.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:26 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:26.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:26 smithi196 ceph-mon[30516]: purged_snaps scrub starts 2024-02-05T06:26:26.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:26 smithi196 ceph-mon[30516]: purged_snaps scrub ok 2024-02-05T06:26:26.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:26 smithi196 ceph-mon[30516]: osd.0 [v2:172.21.15.122:6802/3979415912,v1:172.21.15.122:6803/3979415912] boot 2024-02-05T06:26:26.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:26 smithi196 ceph-mon[30516]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T06:26:26.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:26 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:26:26.895 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: Detected new or changed devices on smithi122 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: Adjusting osd_memory_target on smithi122 to 17088M 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:27.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:27.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T06:26:27.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:27.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: Detected new or changed devices on smithi122 2024-02-05T06:26:27.769 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: Adjusting osd_memory_target on smithi122 to 17088M 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:27.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T06:26:27.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: Detected new or changed devices on smithi122 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: Adjusting osd_memory_target on smithi122 to 17088M 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:28.981 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:28.981 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:28.981 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:28 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:29.019 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:28 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:29.019 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:28 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:29.019 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:28 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:29.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:28 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:29.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:28 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:29.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:28 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:29.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:29 smithi142 ceph-mon[30053]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:29.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:29 smithi142 ceph-mon[30053]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:30.019 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:29 smithi196 ceph-mon[30516]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:30.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:29 smithi196 ceph-mon[30516]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:30.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:29 smithi122 ceph-mon[25512]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:30.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:29 smithi122 ceph-mon[25512]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:32.470 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:32.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: from='client.? 172.21.15.142:0/2364617092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]': finished 2024-02-05T06:26:32.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T06:26:32.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:32 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:32.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:32.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: from='client.? 172.21.15.142:0/2364617092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]': finished 2024-02-05T06:26:32.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T06:26:32.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:32 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:32.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:32.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: from='client.? 172.21.15.142:0/2364617092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]: dispatch 2024-02-05T06:26:32.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "878f12fc-db93-4d3a-8623-62f1fa918bd6"}]': finished 2024-02-05T06:26:32.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T06:26:32.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:32 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:33.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.142:0/3372341398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:33.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:33 smithi142 ceph-mon[30053]: from='client.? 172.21.15.142:0/3372341398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:33.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:33 smithi122 ceph-mon[25512]: from='client.? 172.21.15.142:0/3372341398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:34.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:34 smithi196 ceph-mon[30516]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:34.559 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:34 smithi142 ceph-mon[30053]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:34.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:34 smithi122 ceph-mon[25512]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:36.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:36 smithi142 ceph-mon[30053]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:36.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:36 smithi196 ceph-mon[30516]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:36.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:36 smithi122 ceph-mon[25512]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:38.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:38 smithi196 ceph-mon[30516]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:38.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T06:26:38.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:38 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:38.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:38 smithi142 ceph-mon[30053]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:38.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:38 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T06:26:38.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:38 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:38.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:38 smithi122 ceph-mon[25512]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:38.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T06:26:38.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:38 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:39.302 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:39 smithi142 ceph-mon[30053]: Deploying daemon osd.1 on smithi142 2024-02-05T06:26:39.381 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:39 smithi122 ceph-mon[25512]: Deploying daemon osd.1 on smithi142 2024-02-05T06:26:39.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:39 smithi196 ceph-mon[30516]: Deploying daemon osd.1 on smithi142 2024-02-05T06:26:40.218 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:40 smithi142 ceph-mon[30053]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:40.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:40 smithi122 ceph-mon[25512]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:40.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:40 smithi196 ceph-mon[30516]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:42.257 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:42 smithi142 ceph-mon[30053]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:42.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:42 smithi196 ceph-mon[30516]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:42.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:42 smithi122 ceph-mon[25512]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:44.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:44 smithi142 ceph-mon[30053]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:44.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:44 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:44 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:44 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:44.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:44 smithi196 ceph-mon[30516]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:44.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:44 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:44 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:44 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:44.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:44 smithi122 ceph-mon[25512]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:44.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:44 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:44 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:44.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:44 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:46.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:46 smithi196 ceph-mon[30516]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:46.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:46 smithi142 ceph-mon[30053]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:46.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:46 smithi122 ceph-mon[25512]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:46.846 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-02-05T06:26:47.373 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.1.service 2024-02-05T06:26:47.375 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/nvme4n1... 2024-02-05T06:26:47.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-05T06:26:47.399 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:47.400 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:47 smithi142 ceph-mon[30053]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:47.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:47.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:47 smithi196 ceph-mon[30516]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:47.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:26:47.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:47 smithi122 ceph-mon[25512]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:47.618 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:26:48.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:48 smithi196 ceph-mon[30516]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T06:26:48.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:48 smithi196 ceph-mon[30516]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:48.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:48 smithi196 ceph-mon[30516]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T06:26:48.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:48 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:48.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:48 smithi196 ceph-mon[30516]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:48.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-mon[30053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T06:26:48.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-mon[30053]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:48.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-mon[30053]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T06:26:48.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:48.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-mon[30053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:48.560 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1[34609]: 2024-02-05T06:26:48.286+0000 7f20e2633700 -1 osd.1 0 waiting for initial osdmap 2024-02-05T06:26:48.561 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:26:48 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1[34609]: 2024-02-05T06:26:48.289+0000 7f20de42d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T06:26:48.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:48 smithi122 ceph-mon[25512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T06:26:48.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:48 smithi122 ceph-mon[25512]: from='osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:48.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:48 smithi122 ceph-mon[25512]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T06:26:48.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:48.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:48 smithi122 ceph-mon[25512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme4n1 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0165923 s, 632 MB/s 2024-02-05T06:26:49.525 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-02-05T06:26:49.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:49 smithi196 ceph-mon[30516]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-05T06:26:49.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:49 smithi196 ceph-mon[30516]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T06:26:49.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:49 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.549 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:49 smithi196 ceph-mon[30516]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:49 smithi142 ceph-mon[30053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-05T06:26:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:49 smithi142 ceph-mon[30053]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T06:26:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:49 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:49 smithi142 ceph-mon[30053]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:49.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:49 smithi122 ceph-mon[25512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-05T06:26:49.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:49 smithi122 ceph-mon[25512]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T06:26:49.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:49 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:49.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:49 smithi122 ceph-mon[25512]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T06:26:50.147 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi196:/dev/nvme4n1 2024-02-05T06:26:50.381 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:26:50.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:50 smithi142 ceph-mon[30053]: purged_snaps scrub starts 2024-02-05T06:26:50.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:50 smithi142 ceph-mon[30053]: purged_snaps scrub ok 2024-02-05T06:26:50.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:50 smithi142 ceph-mon[30053]: osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038] boot 2024-02-05T06:26:50.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:50 smithi142 ceph-mon[30053]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T06:26:50.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:50 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:50.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:50 smithi196 ceph-mon[30516]: purged_snaps scrub starts 2024-02-05T06:26:50.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:50 smithi196 ceph-mon[30516]: purged_snaps scrub ok 2024-02-05T06:26:50.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:50 smithi196 ceph-mon[30516]: osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038] boot 2024-02-05T06:26:50.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:50 smithi196 ceph-mon[30516]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T06:26:50.504 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:50 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:50.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:50 smithi122 ceph-mon[25512]: purged_snaps scrub starts 2024-02-05T06:26:50.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:50 smithi122 ceph-mon[25512]: purged_snaps scrub ok 2024-02-05T06:26:50.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:50 smithi122 ceph-mon[25512]: osd.1 [v2:172.21.15.142:6800/1680982038,v1:172.21.15.142:6801/1680982038] boot 2024-02-05T06:26:50.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:50 smithi122 ceph-mon[25512]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T06:26:50.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:50 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: Detected new or changed devices on smithi142 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: Adjusting osd_memory_target on smithi142 to 17088M 2024-02-05T06:26:51.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:51.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:51.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:51 smithi196 ceph-mon[30516]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:51.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T06:26:51.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: Detected new or changed devices on smithi142 2024-02-05T06:26:51.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: Adjusting osd_memory_target on smithi142 to 17088M 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:51 smithi142 ceph-mon[30053]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: Detected new or changed devices on smithi142 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: Adjusting osd_memory_target on smithi142 to 17088M 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:51.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:26:51.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:26:51.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:51 smithi122 ceph-mon[25512]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:52.925 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:52.926 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:52.926 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:52 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:53.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:53.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:53.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:52 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:53.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T06:26:53.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T06:26:53.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:52 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:26:53.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:53 smithi196 ceph-mon[30516]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:53.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:53 smithi196 ceph-mon[30516]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:54.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:53 smithi142 ceph-mon[30053]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:54.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:53 smithi142 ceph-mon[30053]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:54.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:53 smithi122 ceph-mon[25512]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:26:54.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:53 smithi122 ceph-mon[25512]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:55.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:54 smithi196 ceph-mon[30516]: from='client.? 172.21.15.196:0/3863667255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:54 smithi196 ceph-mon[30516]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:54 smithi196 ceph-mon[30516]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]': finished 2024-02-05T06:26:55.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:54 smithi196 ceph-mon[30516]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T06:26:55.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:54 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:26:55.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:54 smithi142 ceph-mon[30053]: from='client.? 172.21.15.196:0/3863667255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:54 smithi142 ceph-mon[30053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:54 smithi142 ceph-mon[30053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]': finished 2024-02-05T06:26:55.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:54 smithi142 ceph-mon[30053]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T06:26:55.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:54 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:26:55.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:54 smithi122 ceph-mon[25512]: from='client.? 172.21.15.196:0/3863667255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:54 smithi122 ceph-mon[25512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]: dispatch 2024-02-05T06:26:55.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:54 smithi122 ceph-mon[25512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52744fbe-6880-48f6-acff-91a387d86631"}]': finished 2024-02-05T06:26:55.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:54 smithi122 ceph-mon[25512]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T06:26:55.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:54 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:26:56.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:56 smithi196 ceph-mon[30516]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:56.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:56 smithi196 ceph-mon[30516]: from='client.? 172.21.15.196:0/3247713930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:56.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:56 smithi142 ceph-mon[30053]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:56.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:56 smithi142 ceph-mon[30053]: from='client.? 172.21.15.196:0/3247713930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:56.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:56 smithi122 ceph-mon[25512]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:56.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:56 smithi122 ceph-mon[25512]: from='client.? 172.21.15.196:0/3247713930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T06:26:58.381 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:26:58 smithi122 ceph-mon[25512]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:58.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:26:58 smithi196 ceph-mon[30516]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:26:58.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:26:58 smithi142 ceph-mon[30053]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:00.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:00 smithi196 ceph-mon[30516]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:00.362 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:00 smithi142 ceph-mon[30053]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:00.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:00 smithi122 ceph-mon[25512]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:02 smithi196 ceph-mon[30516]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T06:27:02.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:02 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:02.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:02 smithi142 ceph-mon[30053]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:02.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T06:27:02.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:02 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:02.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:02 smithi122 ceph-mon[25512]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:02.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T06:27:02.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:02 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:03.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:03 smithi196 ceph-mon[30516]: Deploying daemon osd.2 on smithi196 2024-02-05T06:27:03.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:03 smithi142 ceph-mon[30053]: Deploying daemon osd.2 on smithi196 2024-02-05T06:27:03.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:03 smithi122 ceph-mon[25512]: Deploying daemon osd.2 on smithi196 2024-02-05T06:27:04.209 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:04 smithi196 ceph-mon[30516]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:04.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:04 smithi142 ceph-mon[30053]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:04.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:04 smithi122 ceph-mon[25512]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:06.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:06 smithi196 ceph-mon[30516]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:06.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:06 smithi142 ceph-mon[30053]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:06.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:06 smithi122 ceph-mon[25512]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:08.123 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:08 smithi196 ceph-mon[30516]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:08.123 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:08 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.123 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:08 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.123 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:08 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:08.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:08 smithi142 ceph-mon[30053]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:08.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:08 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:08 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:08 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:08.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:08 smithi122 ceph-mon[25512]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:08.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:08 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:08 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:08.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:08 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:10.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:10 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:10.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:10.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:10.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:10 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:10.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:10.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:10.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:10.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:10 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.092 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-02-05T06:27:11.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:11 smithi196 ceph-mon[30516]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:11 smithi196 ceph-mon[30516]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:11 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:11 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:11 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:11.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:11 smithi142 ceph-mon[30053]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:11 smithi142 ceph-mon[30053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:11 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:11 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:11 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:11.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:11 smithi122 ceph-mon[25512]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:11 smithi122 ceph-mon[25512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T06:27:11.579 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:11 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:11 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:11.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:11 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:11.686 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.2.service 2024-02-05T06:27:11.688 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-05T06:27:11.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-05T06:27:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:12.250 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:12.251 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T06:27:12.251 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T06:27:12.251 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:12.251 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:12.251 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-mon[30516]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:12.251 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2[35820]: 2024-02-05T06:27:12.145+0000 7f66affce700 -1 osd.2 0 waiting for initial osdmap 2024-02-05T06:27:12.251 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:12 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2[35820]: 2024-02-05T06:27:12.147+0000 7f66abdc8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:12.295 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:12 smithi122 ceph-mon[25512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:12.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:12.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T06:27:12.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T06:27:12.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:12.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: from='osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:12.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:12 smithi142 ceph-mon[30053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-02-05T06:27:13.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:13 smithi196 ceph-mon[30516]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-05T06:27:13.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:13 smithi196 ceph-mon[30516]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T06:27:13.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:13 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:13 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:13 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.308 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:13 smithi122 ceph-mon[25512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-05T06:27:13.308 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:13 smithi122 ceph-mon[25512]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T06:27:13.308 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:13 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.308 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:13 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.308 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:13 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.547 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:27:13.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:13 smithi142 ceph-mon[30053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-02-05T06:27:13.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:13 smithi142 ceph-mon[30053]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T06:27:13.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:13 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:13 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:13.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:13 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:14.122 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1707114433,"num_in_osds":3,"osd_in_since":1707114414,"num_remapped_pgs":0} 2024-02-05T06:27:14.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T06:27:14.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: purged_snaps scrub starts 2024-02-05T06:27:14.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: purged_snaps scrub ok 2024-02-05T06:27:14.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:14.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624] boot 2024-02-05T06:27:14.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T06:27:14.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:14.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:14 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/236831805' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T06:27:14.358 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: purged_snaps scrub starts 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: purged_snaps scrub ok 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624] boot 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:14.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:14 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/236831805' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T06:27:14.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: purged_snaps scrub starts 2024-02-05T06:27:14.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: purged_snaps scrub ok 2024-02-05T06:27:14.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T06:27:14.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: osd.2 [v2:172.21.15.196:6800/2473343624,v1:172.21.15.196:6801/2473343624] boot 2024-02-05T06:27:14.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T06:27:14.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:14.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:14 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/236831805' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T06:27:15.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.410 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:15 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:15.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:15 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:15.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:15.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:15 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:16.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:27:16.074 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":20,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","created":"2024-02-05T06:23:54.951169+0000","modified":"2024-02-05T06:27:15.855371+0000","last_up_change":"2024-02-05T06:27:13.147133+0000","last_in_change":"2024-02-05T06:26:54.818304+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T06:27:15.268838+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dda59134-6703-45f2-af25-a75ef3696bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6803","nonce":3979415912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6805","nonce":3979415912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6809","nonce":3979415912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6807","nonce":3979415912}]},"public_addr":"172.21.15.122:6803/3979415912","cluster_addr":"172.21.15.122:6805/3979415912","heartbeat_back_addr":"172.21.15.122:6809/3979415912","heartbeat_front_addr":"172.21.15.122:6807/3979415912","state":["exists","up"]},{"osd":1,"uuid":"878f12fc-db93-4d3a-8623-62f1fa918bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6801","nonce":1680982038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6803","nonce":1680982038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6807","nonce":1680982038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6805","nonce":1680982038}]},"public_addr":"172.21.15.142:6801/1680982038","cluster_addr":"172.21.15.142:6803/1680982038","heartbeat_back_addr":"172.21.15.142:6807/1680982038","heartbeat_front_addr":"172.21.15.142:6805/1680982038","state":["exists","up"]},{"osd":2,"uuid":"52744fbe-6880-48f6-acff-91a387d86631","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6801","nonce":2473343624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6803","nonce":2473343624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6807","nonce":2473343624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6805","nonce":2473343624}]},"public_addr":"172.21.15.196:6801/2473343624","cluster_addr":"172.21.15.196:6803/2473343624","heartbeat_back_addr":"172.21.15.196:6807/2473343624","heartbeat_front_addr":"172.21.15.196:6805/2473343624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:23.764911+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:47.508102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:27:11.314860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/1349799131":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/238815522":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/3566035456":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/1584277443":"2024-02-06T06:24:48.935059+0000","172.21.15.122:0/3677452393":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/857245561":"2024-02-06T06:24:21.886998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T06:27:16.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: Detected new or changed devices on smithi196 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: Adjusting osd_memory_target on smithi196 to 17088M 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:16.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:16 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/4193047900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:16.560 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: Detected new or changed devices on smithi196 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: Adjusting osd_memory_target on smithi196 to 17088M 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:16.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:16 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/4193047900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:16.649 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-05T06:27:15.268838+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-05T06:27:16.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-05T06:27:16.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: Detected new or changed devices on smithi196 2024-02-05T06:27:16.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: Adjusting osd_memory_target on smithi196 to 17088M 2024-02-05T06:27:16.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: pgmap v82: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:16.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:16.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T06:27:16.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T06:27:16.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:16 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/4193047900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:16.883 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:17.293 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46895]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-05T06:27:17.293 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:17.293 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46895]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:17.293 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46895]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:17.579 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46917]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-05T06:27:17.579 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:17.580 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46917]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:17.580 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:27:17 smithi122 sudo[46917]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:17.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:17.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:17 smithi122 ceph-mon[25512]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T06:27:17.906 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:17 smithi122 ceph-mon[25512]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:17.906 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:17 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:17.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:17 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:17.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:17 smithi142 ceph-mon[30053]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T06:27:17.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:17 smithi142 ceph-mon[30053]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:17.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:17 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:17.968 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39495]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-05T06:27:17.968 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:17.968 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39495]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:17.968 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39495]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:18.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:17 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T06:27:18.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:17 smithi196 ceph-mon[30516]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T06:27:18.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:17 smithi196 ceph-mon[30516]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:18.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:17 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:18.310 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39498]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-05T06:27:18.310 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:18.311 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39498]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:18.311 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:27:17 smithi142 sudo[39498]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:18.488 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40628]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-05T06:27:18.488 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:18.488 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40628]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:18.488 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40628]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:18.770 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40631]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-05T06:27:18.770 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:18.770 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40631]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:18.771 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:27:18 smithi196 sudo[40631]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:19.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:18 smithi122 sudo[46996]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T06:27:19.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:18 smithi122 sudo[46996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:19.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:18 smithi122 sudo[46996]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:19.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:18 smithi122 sudo[46996]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:19.015 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-02-05T06:27:19.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 ceph-mon[30516]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T06:27:19.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 ceph-mon[30516]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-05T06:27:19.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:19.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 ceph-mon[30053]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T06:27:19.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 ceph-mon[30053]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-05T06:27:19.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:19.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 sudo[46999]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T06:27:19.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 sudo[46999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 sudo[46999]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 sudo[46999]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 ceph-mon[25512]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 ceph-mon[25512]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:19.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:19 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:19.530 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40634]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T06:27:19.530 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:19.530 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40634]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:19.530 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40634]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:19.664 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi122 2024-02-05T06:27:19.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi122=foo.a' 2024-02-05T06:27:19.901 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:20.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40637]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T06:27:20.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40637]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:20.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40637]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:20.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:19 smithi196 sudo[40637]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:20.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 sudo[39501]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T06:27:20.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 sudo[39501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:20.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 sudo[39501]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:20.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:19 smithi142 sudo[39501]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 sudo[39504]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 sudo[39504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 sudo[39504]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 sudo[39504]: pam_unix(sudo:session): session closed for user root 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:20.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/381234818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:20 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/381234818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.322 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.323 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:20 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/381234818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:20.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:20.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:20.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:20.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:20 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:21.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:21 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:21.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:21 smithi196 ceph-mon[30516]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:21.270 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:21 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:21.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:21 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:21.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:21 smithi142 ceph-mon[30053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:21.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:21 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:21.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:21 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T06:27:21.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:21 smithi122 ceph-mon[25512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T06:27:21.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:21 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:21.752 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled rgw.foo.a update... 2024-02-05T06:27:22.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:22 smithi142 ceph-mon[30053]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:22.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:22 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:22.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:22 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:22.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:22 smithi122 ceph-mon[25512]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:22.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:22 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:22.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:22 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:22.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:22 smithi196 ceph-mon[30516]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:22.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:22 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:22.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:22 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:22.406 DEBUG:teuthology.orchestra.run.smithi122:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-05T06:27:22.408 INFO:tasks.cephadm:Adding prometheus.a on smithi142 2024-02-05T06:27:22.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi142=a' 2024-02-05T06:27:22.640 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:23 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:23.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:23 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:23.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:23.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:23.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:23 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:24.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:24 smithi196 ceph-mon[30516]: Deploying daemon rgw.foo.a on smithi122 2024-02-05T06:27:24.149 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:24 smithi196 ceph-mon[30516]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:24.280 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:24 smithi122 ceph-mon[25512]: Deploying daemon rgw.foo.a on smithi122 2024-02-05T06:27:24.280 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:24 smithi122 ceph-mon[25512]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:24.310 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:24 smithi142 ceph-mon[30053]: Deploying daemon rgw.foo.a on smithi122 2024-02-05T06:27:24.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:24 smithi142 ceph-mon[30053]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:24.376 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-02-05T06:27:24.580 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:24 smithi122 systemd[1]: Starting Ceph rgw.foo.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:27:24.932 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:24 smithi122 podman[47329]: 2024-02-05 06:27:24.576340885 +0000 UTC m=+0.017199891 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:27:24.933 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:24 smithi122 podman[47329]: 2024-02-05 06:27:24.702798969 +0000 UTC m=+0.143658011 container create 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-05T06:27:24.933 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:24 smithi122 podman[47329]: 2024-02-05 06:27:24.938623236 +0000 UTC m=+0.379482236 container init 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-02-05T06:27:24.963 DEBUG:teuthology.orchestra.run.smithi142:prometheus.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-05T06:27:24.966 INFO:tasks.cephadm:Adding node-exporter.a on smithi122 2024-02-05T06:27:24.966 INFO:tasks.cephadm:Adding node-exporter.b on smithi142 2024-02-05T06:27:24.967 INFO:tasks.cephadm:Adding node-exporter.c on smithi196 2024-02-05T06:27:24.967 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '3;smithi122=a;smithi142=b;smithi196=c' 2024-02-05T06:27:25.196 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:25.330 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:24 smithi122 podman[47329]: 2024-02-05 06:27:24.944021049 +0000 UTC m=+0.384880049 container start 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-05T06:27:25.330 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:25 smithi122 bash[47329]: 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 2024-02-05T06:27:25.330 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:27:25 smithi122 systemd[1]: Started Ceph rgw.foo.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:27:25.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:25.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: Saving service prometheus spec with placement smithi142=a;count:1 2024-02-05T06:27:25.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:25.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:25 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: Saving service prometheus spec with placement smithi142=a;count:1 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:25 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: Saving service prometheus spec with placement smithi142=a;count:1 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:25.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:25 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:26.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:26 smithi122 ceph-mon[25512]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:26.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:26 smithi122 ceph-mon[25512]: osdmap e23: 3 total, 3 up, 3 in 2024-02-05T06:27:26.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:26 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T06:27:26.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:26 smithi196 ceph-mon[30516]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:26.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:26 smithi196 ceph-mon[30516]: osdmap e23: 3 total, 3 up, 3 in 2024-02-05T06:27:26.647 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:26 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T06:27:26.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:26 smithi142 ceph-mon[30053]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-02-05T06:27:26.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:26 smithi142 ceph-mon[30053]: osdmap e23: 3 total, 3 up, 3 in 2024-02-05T06:27:26.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:26 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T06:27:27.200 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-02-05T06:27:27.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T06:27:27.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: osdmap e24: 3 total, 3 up, 3 in 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: Checking dashboard <-> RGW credentials 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: pgmap v93: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi122=a;smithi142=b;smithi196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: Saving service node-exporter spec with placement smithi122=a;smithi142=b;smithi196=c;count:3 2024-02-05T06:27:27.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:27 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: osdmap e24: 3 total, 3 up, 3 in 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: Checking dashboard <-> RGW credentials 2024-02-05T06:27:27.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: pgmap v93: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:27.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi122=a;smithi142=b;smithi196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:27.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: Saving service node-exporter spec with placement smithi122=a;smithi142=b;smithi196=c;count:3 2024-02-05T06:27:27.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:27 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.817 DEBUG:teuthology.orchestra.run.smithi122:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-05T06:27:27.819 DEBUG:teuthology.orchestra.run.smithi142:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-05T06:27:27.822 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.c> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.c.service 2024-02-05T06:27:27.824 INFO:tasks.cephadm:Adding alertmanager.a on smithi122 2024-02-05T06:27:27.825 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi122=a' 2024-02-05T06:27:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1022757859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T06:27:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: osdmap e24: 3 total, 3 up, 3 in 2024-02-05T06:27:27.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: Checking dashboard <-> RGW credentials 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: pgmap v93: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi122=a;smithi142=b;smithi196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: Saving service node-exporter spec with placement smithi122=a;smithi142=b;smithi196=c;count:3 2024-02-05T06:27:27.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:27 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:28.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:28 smithi196 ceph-mon[30516]: osdmap e25: 3 total, 3 up, 3 in 2024-02-05T06:27:28.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:28 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:28.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:28 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:28.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:28 smithi122 ceph-mon[25512]: osdmap e25: 3 total, 3 up, 3 in 2024-02-05T06:27:28.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:28 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:28.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:28 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:28.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:28 smithi142 ceph-mon[30053]: osdmap e25: 3 total, 3 up, 3 in 2024-02-05T06:27:28.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:28 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:28.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:28 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T06:27:29.494 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:29 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.494 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:29 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.494 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:29 smithi196 ceph-mon[30516]: osdmap e26: 3 total, 3 up, 3 in 2024-02-05T06:27:29.494 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:29 smithi196 ceph-mon[30516]: pgmap v96: 65 pgs: 56 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:29.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:29 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:29 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:29 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:29 smithi142 ceph-mon[30053]: osdmap e26: 3 total, 3 up, 3 in 2024-02-05T06:27:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:29 smithi142 ceph-mon[30053]: pgmap v96: 65 pgs: 56 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:29 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:29.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:29 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:29 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T06:27:29.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:29 smithi122 ceph-mon[25512]: osdmap e26: 3 total, 3 up, 3 in 2024-02-05T06:27:29.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:29 smithi122 ceph-mon[25512]: pgmap v96: 65 pgs: 56 unknown, 9 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:29.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:29 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:29.946 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-02-05T06:27:30.520 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: osdmap e27: 3 total, 3 up, 3 in 2024-02-05T06:27:30.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:30.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-02-05T06:27:30.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:30 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:30.627 DEBUG:teuthology.orchestra.run.smithi122:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-05T06:27:30.630 INFO:tasks.cephadm:Adding grafana.a on smithi142 2024-02-05T06:27:30.630 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi142=a' 2024-02-05T06:27:30.712 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: osdmap e27: 3 total, 3 up, 3 in 2024-02-05T06:27:30.712 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.712 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.712 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:30.713 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-02-05T06:27:30.713 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:30 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: osdmap e27: 3 total, 3 up, 3 in 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-02-05T06:27:30.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:30 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:30.878 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:31.750 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:31 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.751 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:31 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.751 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:31 smithi196 ceph-mon[30516]: osdmap e28: 3 total, 3 up, 3 in 2024-02-05T06:27:31.751 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:31 smithi196 ceph-mon[30516]: pgmap v99: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-05T06:27:31.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:31 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:31 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:31 smithi142 ceph-mon[30053]: osdmap e28: 3 total, 3 up, 3 in 2024-02-05T06:27:31.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:31 smithi142 ceph-mon[30053]: pgmap v99: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-05T06:27:31.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:31 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:31 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T06:27:31.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:31 smithi122 ceph-mon[25512]: osdmap e28: 3 total, 3 up, 3 in 2024-02-05T06:27:31.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:31 smithi122 ceph-mon[25512]: pgmap v99: 97 pgs: 9 creating+peering, 53 active+clean, 35 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-05T06:27:32.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:32 smithi196 ceph-mon[30516]: osdmap e29: 3 total, 3 up, 3 in 2024-02-05T06:27:32.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:32 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:32.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:32 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:32.777 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-02-05T06:27:32.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:32 smithi142 ceph-mon[30053]: osdmap e29: 3 total, 3 up, 3 in 2024-02-05T06:27:32.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:32 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:32.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:32 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:32 smithi122 ceph-mon[25512]: osdmap e29: 3 total, 3 up, 3 in 2024-02-05T06:27:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:32 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:32 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T06:27:33.401 DEBUG:teuthology.orchestra.run.smithi142:grafana.a> sudo journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@grafana.a.service 2024-02-05T06:27:33.404 INFO:tasks.cephadm:Setting up client nodes... 2024-02-05T06:27:33.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T06:27:33.714 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: osdmap e30: 3 total, 3 up, 3 in 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: Saving service grafana spec with placement smithi142=a;count:1 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:33.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:33 smithi196 ceph-mon[30516]: pgmap v102: 129 pgs: 9 creating+peering, 76 active+clean, 44 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: osdmap e30: 3 total, 3 up, 3 in 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: Saving service grafana spec with placement smithi142=a;count:1 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:33.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:33 smithi142 ceph-mon[30053]: pgmap v102: 129 pgs: 9 creating+peering, 76 active+clean, 44 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: osdmap e30: 3 total, 3 up, 3 in 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: Saving service grafana spec with placement smithi142=a;count:1 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:33.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:33 smithi122 ceph-mon[25512]: pgmap v102: 129 pgs: 9 creating+peering, 76 active+clean, 44 unknown; 578 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: osdmap e31: 3 total, 3 up, 3 in 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:34 smithi196 ceph-mon[30516]: Deploying daemon prometheus.a on smithi142 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: osdmap e31: 3 total, 3 up, 3 in 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.802 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:34 smithi122 ceph-mon[25512]: Deploying daemon prometheus.a on smithi142 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/321475920' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/3097758557' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: osdmap e31: 3 total, 3 up, 3 in 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:34.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:34 smithi142 ceph-mon[30053]: Deploying daemon prometheus.a on smithi142 2024-02-05T06:27:35.582 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-02-05T06:27:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDXf8BlXNb3IhAACadIq4rA4/ilkwyb5c5F3w== 2024-02-05T06:27:35.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:35 smithi196 ceph-mon[30516]: pgmap v104: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 579 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 3.5 KiB/s wr, 70 op/s 2024-02-05T06:27:35.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:35 smithi142 ceph-mon[30053]: pgmap v104: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 579 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 3.5 KiB/s wr, 70 op/s 2024-02-05T06:27:35.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:35 smithi122 ceph-mon[25512]: pgmap v104: 129 pgs: 9 creating+peering, 97 active+clean, 23 unknown; 579 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 3.5 KiB/s wr, 70 op/s 2024-02-05T06:27:36.158 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:27:36.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-05T06:27:36.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-05T06:27:36.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T06:27:36.446 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T06:27:36.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:36 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:36.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:36 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:36.770 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:36 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:36.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:36 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:36.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:36 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:36.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:36 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1279947582' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:37.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:37 smithi196 ceph-mon[30516]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.3 KiB/s wr, 188 op/s 2024-02-05T06:27:37.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:37 smithi142 ceph-mon[30053]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.3 KiB/s wr, 188 op/s 2024-02-05T06:27:37.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:37 smithi122 ceph-mon[25512]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 77 KiB/s rd, 6.3 KiB/s wr, 188 op/s 2024-02-05T06:27:39.099 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-02-05T06:27:39.099 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQDbf8Bljl0BBhAA+inoYvr55o70CEt1dwM33g== 2024-02-05T06:27:39.868 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:27:39.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-05T06:27:39.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-05T06:27:39.911 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T06:27:40.155 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.c/config 2024-02-05T06:27:40.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:40 smithi196 ceph-mon[30516]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 61 KiB/s rd, 4.6 KiB/s wr, 149 op/s 2024-02-05T06:27:40.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:40 smithi196 ceph-mon[30516]: from='client.? 172.21.15.142:0/1962510521' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:40 smithi196 ceph-mon[30516]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:40 smithi196 ceph-mon[30516]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:40.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:40 smithi142 ceph-mon[30053]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 61 KiB/s rd, 4.6 KiB/s wr, 149 op/s 2024-02-05T06:27:40.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:40 smithi142 ceph-mon[30053]: from='client.? 172.21.15.142:0/1962510521' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:40 smithi142 ceph-mon[30053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:40 smithi142 ceph-mon[30053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:40.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:40 smithi122 ceph-mon[25512]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 61 KiB/s rd, 4.6 KiB/s wr, 149 op/s 2024-02-05T06:27:40.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:40 smithi122 ceph-mon[25512]: from='client.? 172.21.15.142:0/1962510521' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:40 smithi122 ceph-mon[25512]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:40.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:40 smithi122 ceph-mon[25512]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:41.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:41 smithi196 ceph-mon[30516]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 54 KiB/s rd, 4.1 KiB/s wr, 131 op/s 2024-02-05T06:27:41.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:41 smithi142 ceph-mon[30053]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 54 KiB/s rd, 4.1 KiB/s wr, 131 op/s 2024-02-05T06:27:41.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:41 smithi122 ceph-mon[25512]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 54 KiB/s rd, 4.1 KiB/s wr, 131 op/s 2024-02-05T06:27:42.051 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-02-05T06:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDef8BlVSoyAxAARt9C4Lky0IYTGYH868/MNQ== 2024-02-05T06:27:42.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:42 smithi142 ceph-mon[30053]: from='client.? 172.21.15.196:0/1580783790' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:42 smithi142 ceph-mon[30053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:42 smithi142 ceph-mon[30053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:42.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:42 smithi122 ceph-mon[25512]: from='client.? 172.21.15.196:0/1580783790' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:42 smithi122 ceph-mon[25512]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:42 smithi122 ceph-mon[25512]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:42.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:42 smithi196 ceph-mon[30516]: from='client.? 172.21.15.196:0/1580783790' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:42 smithi196 ceph-mon[30516]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T06:27:42.651 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:42 smithi196 ceph-mon[30516]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T06:27:42.676 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:27:42.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-05T06:27:42.677 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-05T06:27:42.717 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-05T06:27:42.717 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-05T06:27:42.718 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-05T06:27:42.960 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:43.360 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:43 smithi122 ceph-mon[25512]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T06:27:43.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:43 smithi142 ceph-mon[30053]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T06:27:43.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:43 smithi196 ceph-mon[30516]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T06:27:44.790 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:27:45.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:45 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:45.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:45 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/2902776198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T06:27:45.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:45 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:45.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:45 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/2902776198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T06:27:45.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:45 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:45.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:45 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/2902776198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T06:27:45.429 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1482012253},{"type":"v1","addr":"172.21.15.122:6801","nonce":1482012253}]},"active_addr":"172.21.15.122:6801/1482012253","active_change":"2024-02-05T06:24:48.935165+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.122:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2030682935}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":366135322}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":740074266}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":4051785213}]}]} 2024-02-05T06:27:45.435 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-05T06:27:45.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-05T06:27:45.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T06:27:45.666 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:46.039 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:46 smithi122 ceph-mon[25512]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 40 KiB/s rd, 3.0 KiB/s wr, 97 op/s 2024-02-05T06:27:46.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-mon[30053]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 40 KiB/s rd, 3.0 KiB/s wr, 97 op/s 2024-02-05T06:27:46.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:46 smithi196 ceph-mon[30516]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 40 KiB/s rd, 3.0 KiB/s wr, 97 op/s 2024-02-05T06:27:46.441 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 systemd[1]: Starting Ceph prometheus.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 podman[40421]: 2024-02-05 06:27:46.446716042 +0000 UTC m=+0.103904685 container create 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 podman[40421]: 2024-02-05 06:27:46.361238223 +0000 UTC m=+0.018426820 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 podman[40421]: 2024-02-05 06:27:46.624003978 +0000 UTC m=+0.281192576 container init 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 podman[40421]: 2024-02-05 06:27:46.628961751 +0000 UTC m=+0.286150351 container start 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.651Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.652Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.652Z caller=main.go:570 level=info host_details="(Linux 5.14.0-416.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024 x86_64 smithi142 (none))" 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.652Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-05T06:27:46.812 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.652Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.653Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.653Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.655Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.655Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.656Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.656Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.455µs 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.656Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.656Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.656Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=15.844µs wal_replay_duration=344.151µs wbl_replay_duration=102ns total_replay_duration=377.127µs 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.657Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.657Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.657Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-05T06:27:46.813 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.674Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.647167ms db_storage=824ns remote_storage=1.091µs web_handler=349ns query_engine=495ns scrape=389.458µs scrape_sd=75.057µs notify=713ns notify_sd=1.292µs rules=16.004183ms tracing=5.47µs 2024-02-05T06:27:46.814 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.674Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-05T06:27:46.814 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:27:46.674Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-05T06:27:46.814 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 bash[40421]: 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 2024-02-05T06:27:46.814 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:27:46 smithi142 systemd[1]: Started Ceph prometheus.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:27:47.499 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:27:47.500 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":31,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","created":"2024-02-05T06:23:54.951169+0000","modified":"2024-02-05T06:27:33.438621+0000","last_up_change":"2024-02-05T06:27:13.147133+0000","last_in_change":"2024-02-05T06:26:54.818304+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T06:27:15.268838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-05T06:27:25.215966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-05T06:27:26.695390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-05T06:27:28.654466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-05T06:27:30.724223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dda59134-6703-45f2-af25-a75ef3696bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6803","nonce":3979415912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6805","nonce":3979415912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6809","nonce":3979415912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6807","nonce":3979415912}]},"public_addr":"172.21.15.122:6803/3979415912","cluster_addr":"172.21.15.122:6805/3979415912","heartbeat_back_addr":"172.21.15.122:6809/3979415912","heartbeat_front_addr":"172.21.15.122:6807/3979415912","state":["exists","up"]},{"osd":1,"uuid":"878f12fc-db93-4d3a-8623-62f1fa918bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6801","nonce":1680982038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6803","nonce":1680982038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6807","nonce":1680982038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6805","nonce":1680982038}]},"public_addr":"172.21.15.142:6801/1680982038","cluster_addr":"172.21.15.142:6803/1680982038","heartbeat_back_addr":"172.21.15.142:6807/1680982038","heartbeat_front_addr":"172.21.15.142:6805/1680982038","state":["exists","up"]},{"osd":2,"uuid":"52744fbe-6880-48f6-acff-91a387d86631","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6801","nonce":2473343624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6803","nonce":2473343624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6807","nonce":2473343624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6805","nonce":2473343624}]},"public_addr":"172.21.15.196:6801/2473343624","cluster_addr":"172.21.15.196:6803/2473343624","heartbeat_back_addr":"172.21.15.196:6807/2473343624","heartbeat_front_addr":"172.21.15.196:6805/2473343624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:23.764911+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:47.508102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:27:11.314860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/1349799131":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/238815522":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/3566035456":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/1584277443":"2024-02-06T06:24:48.935059+0000","172.21.15.122:0/3677452393":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/857245561":"2024-02-06T06:24:21.886998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 28 KiB/s rd, 1.8 KiB/s wr, 67 op/s 2024-02-05T06:27:47.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/948697002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:48.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T06:27:48.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 28 KiB/s rd, 1.8 KiB/s wr, 67 op/s 2024-02-05T06:27:48.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/948697002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:48.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: ignoring --setuser ceph since I am not root 2024-02-05T06:27:48.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: ignoring --setgroup ceph since I am not root 2024-02-05T06:27:48.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:47 smithi196 ceph-mgr[32599]: -- 172.21.15.196:0/446033853 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56465d4a3520 con 0x56465d479000 2024-02-05T06:27:48.061 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: ignoring --setuser ceph since I am not root 2024-02-05T06:27:48.061 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: ignoring --setgroup ceph since I am not root 2024-02-05T06:27:48.061 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mgr[31530]: -- 172.21.15.142:0/1793047846 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559d89f9520 con 0x5559d89ce800 2024-02-05T06:27:48.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' 2024-02-05T06:27:48.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T06:27:48.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 28 KiB/s rd, 1.8 KiB/s wr, 67 op/s 2024-02-05T06:27:48.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:47 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/948697002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:48.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setuser ceph since I am not root 2024-02-05T06:27:48.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:47 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ignoring --setgroup ceph since I am not root 2024-02-05T06:27:48.129 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-05T06:27:48.130 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T06:27:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:48.639 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:48.371+0000 7f25d907b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:27:48.639 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:48 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:48.477+0000 7f25d907b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:27:48.771 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:48 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:48.350+0000 7f8b5345a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:27:48.771 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:48 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:48.443+0000 7f8b5345a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:27:48.787 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:48 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:48.355+0000 7f831ce44200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T06:27:48.788 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:48 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:48.447+0000 7f831ce44200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T06:27:48.902 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:48 smithi122 ceph-mon[25512]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T06:27:48.902 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:48 smithi122 ceph-mon[25512]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-05T06:27:49.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:48 smithi142 ceph-mon[30053]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T06:27:49.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:48 smithi142 ceph-mon[30053]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-05T06:27:49.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:48 smithi196 ceph-mon[30516]: from='mgr.14152 172.21.15.122:0/2260210595' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T06:27:49.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:48 smithi196 ceph-mon[30516]: mgrmap e17: y(active, since 2m), standbys: z, x 2024-02-05T06:27:49.212 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.122+0000 7f25d907b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:27:49.255 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.216+0000 7f25d907b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:27:49.398 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.077+0000 7f831ce44200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:27:49.399 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.168+0000 7f831ce44200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:27:49.399 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.308+0000 7f831ce44200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:27:49.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.078+0000 7f8b5345a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T06:27:49.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.170+0000 7f8b5345a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T06:27:49.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.312+0000 7f8b5345a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:27:49.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.407+0000 7f8b5345a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:27:49.514 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.359+0000 7f25d907b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T06:27:49.514 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.454+0000 7f25d907b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:27:49.746 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.403+0000 7f831ce44200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T06:27:49.746 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.485+0000 7f831ce44200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:27:49.746 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.751+0000 7f831ce44200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:27:49.749 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.489+0000 7f8b5345a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:27:49.749 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.753+0000 7f8b5345a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:27:49.792 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.537+0000 7f25d907b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T06:27:50.021 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.862+0000 7f8b5345a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:50.021 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:49 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:49.949+0000 7f8b5345a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:27:50.035 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.861+0000 7f831ce44200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:50.035 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:49 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:49.947+0000 7f831ce44200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:27:50.035 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:50 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:50.040+0000 7f831ce44200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:27:50.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.812+0000 7f25d907b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T06:27:50.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:49 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:49.925+0000 7f25d907b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:50.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:50 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:50.015+0000 7f25d907b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T06:27:50.230 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:27:50.230 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":31,"fsid":"f68cd702-c3ee-11ee-95b6-87774f69a715","created":"2024-02-05T06:23:54.951169+0000","modified":"2024-02-05T06:27:33.438621+0000","last_up_change":"2024-02-05T06:27:13.147133+0000","last_in_change":"2024-02-05T06:26:54.818304+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T06:27:15.268838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-05T06:27:25.215966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-05T06:27:26.695390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-05T06:27:28.654466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-05T06:27:30.724223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dda59134-6703-45f2-af25-a75ef3696bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6803","nonce":3979415912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6805","nonce":3979415912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6809","nonce":3979415912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3979415912},{"type":"v1","addr":"172.21.15.122:6807","nonce":3979415912}]},"public_addr":"172.21.15.122:6803/3979415912","cluster_addr":"172.21.15.122:6805/3979415912","heartbeat_back_addr":"172.21.15.122:6809/3979415912","heartbeat_front_addr":"172.21.15.122:6807/3979415912","state":["exists","up"]},{"osd":1,"uuid":"878f12fc-db93-4d3a-8623-62f1fa918bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6801","nonce":1680982038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6803","nonce":1680982038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6807","nonce":1680982038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1680982038},{"type":"v1","addr":"172.21.15.142:6805","nonce":1680982038}]},"public_addr":"172.21.15.142:6801/1680982038","cluster_addr":"172.21.15.142:6803/1680982038","heartbeat_back_addr":"172.21.15.142:6807/1680982038","heartbeat_front_addr":"172.21.15.142:6805/1680982038","state":["exists","up"]},{"osd":2,"uuid":"52744fbe-6880-48f6-acff-91a387d86631","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6801","nonce":2473343624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6803","nonce":2473343624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6807","nonce":2473343624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2473343624},{"type":"v1","addr":"172.21.15.196:6805","nonce":2473343624}]},"public_addr":"172.21.15.196:6801/2473343624","cluster_addr":"172.21.15.196:6803/2473343624","heartbeat_back_addr":"172.21.15.196:6807/2473343624","heartbeat_front_addr":"172.21.15.196:6805/2473343624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:23.764911+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:26:47.508102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T06:27:11.314860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/1349799131":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/238815522":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/3566035456":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/1584277443":"2024-02-06T06:24:48.935059+0000","172.21.15.122:0/3677452393":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6800/476362435":"2024-02-06T06:24:48.935059+0000","172.21.15.122:6801/2435033690":"2024-02-06T06:24:21.886998+0000","172.21.15.122:0/857245561":"2024-02-06T06:24:21.886998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T06:27:50.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:50 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/2602330207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:50.311 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:50 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:50.216+0000 7f831ce44200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:27:50.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:50 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/2602330207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:50.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:50 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:50.044+0000 7f8b5345a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:27:50.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:50 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:50.220+0000 7f8b5345a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:27:50.528 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:50 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/2602330207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T06:27:50.528 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:50 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:50.113+0000 7f25d907b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T06:27:50.528 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:50 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:50.297+0000 7f25d907b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T06:27:50.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-05T06:27:50.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-05T06:27:50.809 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-05T06:27:51.074 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:51.080 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:51.085 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:51.458 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:51.324+0000 7f25d907b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:27:51.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:51 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:51.187+0000 7f8b5345a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:27:51.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:51 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:51.317+0000 7f8b5345a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:27:51.561 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:51 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:51.175+0000 7f831ce44200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T06:27:51.561 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:51 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:51.304+0000 7f831ce44200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:27:51.775 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:51.462+0000 7f25d907b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T06:27:51.920 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:51 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:51.639+0000 7f831ce44200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:27:51.920 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:51 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:51.729+0000 7f831ce44200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:27:51.920 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:51 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:51.924+0000 7f831ce44200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:51.932 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:51 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:51.653+0000 7f8b5345a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:27:51.932 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:51 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:51.743+0000 7f8b5345a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:27:51.932 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:51 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:51.937+0000 7f8b5345a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:52.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:51.811+0000 7f25d907b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T06:27:52.080 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:51.902+0000 7f25d907b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T06:27:52.271 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:52 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:52.120+0000 7f8b5345a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:27:52.311 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:52 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:52.106+0000 7f831ce44200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:27:52.522 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:52 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:52.098+0000 7f25d907b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:52.522 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:52 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:52.282+0000 7f25d907b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T06:27:52.648 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:52 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:52.391+0000 7f831ce44200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:52.648 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:52 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:52.652+0000 7f831ce44200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:52.659 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:52 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:52.403+0000 7f8b5345a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:52.660 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:52 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:52.664+0000 7f8b5345a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:52.813 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:52 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:52.569+0000 7f25d907b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T06:27:53.021 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:52 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:52.898+0000 7f8b5345a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:27:53.061 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:52 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:52.888+0000 7f831ce44200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:27:53.075 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:52 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:52.828+0000 7f25d907b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T06:27:53.076 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:53 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:53.079+0000 7f25d907b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T06:27:54.021 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:53 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:53.690+0000 7f8b5345a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:27:54.057 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:53 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:53.910+0000 7f25d907b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:27:54.061 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:53 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:53.687+0000 7f831ce44200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T06:27:54.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:54.265+0000 7f8b5345a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:27:54.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:54.364+0000 7f8b5345a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:27:54.561 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:54.260+0000 7f831ce44200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:27:54.562 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:54.359+0000 7f831ce44200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:27:54.644 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:54.506+0000 7f25d907b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T06:27:54.644 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:54.607+0000 7f25d907b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T06:27:54.925 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:54.828+0000 7f25d907b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: Standby manager daemon x restarted 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: Standby manager daemon x started 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: Standby manager daemon z restarted 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: Standby manager daemon z started 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:27:54.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:54.927 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:27:54.927 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-mon[25512]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:55.000 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: Standby manager daemon x restarted 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: Standby manager daemon x started 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: Standby manager daemon z restarted 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: Standby manager daemon z started 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:55.001 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-mon[30516]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:54.579+0000 7f8b5345a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: [05/Feb/2024:06:27:54] ENGINE Bus STARTING 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: CherryPy Checker: 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: The Application mounted at '' has an empty config. 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: [05/Feb/2024:06:27:54] ENGINE Serving on http://:::9283 2024-02-05T06:27:55.002 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:27:54 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z[32536]: [05/Feb/2024:06:27:54] ENGINE Bus STARTED 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: Standby manager daemon x restarted 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: Standby manager daemon x started 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T06:27:55.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: Standby manager daemon z restarted 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: Standby manager daemon z started 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.142:0/2359446867' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-mon[30053]: from='mgr.? 172.21.15.196:0/967123054' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:54.575+0000 7f831ce44200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: [05/Feb/2024:06:27:54] ENGINE Bus STARTING 2024-02-05T06:27:55.004 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: CherryPy Checker: 2024-02-05T06:27:55.005 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: The Application mounted at '' has an empty config. 2024-02-05T06:27:55.005 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: 2024-02-05T06:27:55.005 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: [05/Feb/2024:06:27:54] ENGINE Serving on http://:::9283 2024-02-05T06:27:55.005 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:27:54 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x[31507]: [05/Feb/2024:06:27:54] ENGINE Bus STARTED 2024-02-05T06:27:55.330 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:54 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:27:54] ENGINE Bus STARTING 2024-02-05T06:27:55.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: CherryPy Checker: 2024-02-05T06:27:55.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: The Application mounted at '' has an empty config. 2024-02-05T06:27:55.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:27:55.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:27:55] ENGINE Serving on http://:::9283 2024-02-05T06:27:55.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:27:55] ENGINE Bus STARTED 2024-02-05T06:27:55.559 INFO:teuthology.orchestra.run.smithi122.stdout:55834574863 2024-02-05T06:27:55.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-05T06:27:55.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-05T06:27:55.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: Active manager daemon y restarted 2024-02-05T06:27:55.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: Activating manager daemon y 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: osdmap e32: 3 total, 3 up, 3 in 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: mgrmap e19: y(active, starting, since 0.00933783s), standbys: x, z 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: Manager daemon y is now available 2024-02-05T06:27:55.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:55.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:55.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:55.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:27:55.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:27:55.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:55 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:27:55.993 INFO:teuthology.orchestra.run.smithi122.stdout:77309411338 2024-02-05T06:27:55.993 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-05T06:27:56.008 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:56.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-05T06:27:56.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: Active manager daemon y restarted 2024-02-05T06:27:56.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: Activating manager daemon y 2024-02-05T06:27:56.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: osdmap e32: 3 total, 3 up, 3 in 2024-02-05T06:27:56.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: mgrmap e19: y(active, starting, since 0.00933783s), standbys: x, z 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: Manager daemon y is now available 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:56.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:56.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:56.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:27:56.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:27:56.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:55 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:27:56.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-05T06:27:56.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: Active manager daemon y restarted 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: Activating manager daemon y 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: osdmap e32: 3 total, 3 up, 3 in 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: mgrmap e19: y(active, starting, since 0.00933783s), standbys: x, z 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T06:27:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: Manager daemon y is now available 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T06:27:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:55 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: mgrmap e20: y(active, since 1.01562s), standbys: x, z 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: [05/Feb/2024:06:27:56] ENGINE Bus STARTING 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: [05/Feb/2024:06:27:56] ENGINE Serving on http://172.21.15.122:8765 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: [05/Feb/2024:06:27:56] ENGINE Serving on https://172.21.15.122:7150 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: [05/Feb/2024:06:27:56] ENGINE Bus STARTED 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:56.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:56 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: mgrmap e20: y(active, since 1.01562s), standbys: x, z 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: [05/Feb/2024:06:27:56] ENGINE Bus STARTING 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: [05/Feb/2024:06:27:56] ENGINE Serving on http://172.21.15.122:8765 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: [05/Feb/2024:06:27:56] ENGINE Serving on https://172.21.15.122:7150 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: [05/Feb/2024:06:27:56] ENGINE Bus STARTED 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:57.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:56 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: mgrmap e20: y(active, since 1.01562s), standbys: x, z 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: [05/Feb/2024:06:27:56] ENGINE Bus STARTING 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: [05/Feb/2024:06:27:56] ENGINE Serving on http://172.21.15.122:8765 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: [05/Feb/2024:06:27:56] ENGINE Serving on https://172.21.15.122:7150 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: [05/Feb/2024:06:27:56] ENGINE Bus STARTED 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:57.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:56 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:58.056 INFO:teuthology.orchestra.run.smithi122.stdout:34359738389 2024-02-05T06:27:58.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-05T06:27:58.109 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:27:58.859 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:58.860 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:58.860 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:58.860 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:58.860 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:58.860 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:58 smithi122 ceph-mon[25512]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-05T06:27:59.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:59.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:58 smithi196 ceph-mon[30516]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-05T06:27:59.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:27:59.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:58 smithi142 ceph-mon[30053]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-05T06:27:59.400 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ::ffff:172.21.15.142 - - [05/Feb/2024:06:27:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T06:27:59.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:59 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:59 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:59 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:27:59.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:59 smithi196 ceph-mon[30516]: Adjusting osd_memory_target on smithi142 to 16064M 2024-02-05T06:27:59.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:27:59 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:27:59.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:27:59.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-mon[25512]: Adjusting osd_memory_target on smithi142 to 16064M 2024-02-05T06:27:59.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:27:59 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:00.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:59 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:00.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:59 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:00.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:59 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:28:00.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:59 smithi142 ceph-mon[30053]: Adjusting osd_memory_target on smithi142 to 16064M 2024-02-05T06:28:00.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:27:59 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:00.420 INFO:teuthology.orchestra.run.smithi122.stdout:55834574864 2024-02-05T06:28:00.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:00 smithi122 ceph-mon[25512]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:00.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:00 smithi122 ceph-mon[25512]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-05T06:28:00.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:00 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1322334958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T06:28:00.963 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:00 smithi142 ceph-mon[30053]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:00.963 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:00 smithi142 ceph-mon[30053]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-05T06:28:00.964 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:00 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1322334958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T06:28:01.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:00 smithi196 ceph-mon[30516]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:01.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:00 smithi196 ceph-mon[30516]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-05T06:28:01.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:00 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1322334958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T06:28:01.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574863 got 55834574864 for osd.1 2024-02-05T06:28:01.052 DEBUG:teuthology.parallel:result is None 2024-02-05T06:28:01.835 INFO:teuthology.orchestra.run.smithi122.stdout:77309411339 2024-02-05T06:28:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:02.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411338 got 77309411339 for osd.2 2024-02-05T06:28:02.479 DEBUG:teuthology.parallel:result is None 2024-02-05T06:28:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:02 smithi122 ceph-mon[25512]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:02 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/1942169277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T06:28:03.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:02 smithi196 ceph-mon[30516]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:03.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:02 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/1942169277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T06:28:03.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:02 smithi142 ceph-mon[30053]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:03.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:02 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/1942169277' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T06:28:04.296 INFO:teuthology.orchestra.run.smithi122.stdout:34359738390 2024-02-05T06:28:04.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:04 smithi122 ceph-mon[25512]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-05T06:28:04.920 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:04 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:04.920 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:04 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:04.920 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:04 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/282293182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T06:28:04.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738389 got 34359738390 for osd.0 2024-02-05T06:28:04.921 DEBUG:teuthology.parallel:result is None 2024-02-05T06:28:04.921 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-05T06:28:04.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T06:28:05.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:04 smithi196 ceph-mon[30516]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-05T06:28:05.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:04 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:05.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:04 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:05.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:04 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/282293182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T06:28:05.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:04 smithi142 ceph-mon[30053]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-05T06:28:05.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:04 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:05.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:04 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:05.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:04 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/282293182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T06:28:05.188 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:06.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-05T06:28:06.271 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Adjusting osd_memory_target on smithi122 to 16064M 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:28:06.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi122:/etc/ceph/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:05 smithi196 ceph-mon[30516]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-05T06:28:06.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:28:06.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Adjusting osd_memory_target on smithi122 to 16064M 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi122:/etc/ceph/ceph.conf 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:28:06.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.315 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.315 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:05 smithi142 ceph-mon[30053]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-05T06:28:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T06:28:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Adjusting osd_memory_target on smithi122 to 16064M 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi122:/etc/ceph/ceph.conf 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-05T06:28:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi196:/etc/ceph/ceph.conf 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:05 smithi122 ceph-mon[25512]: Updating smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T06:28:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:28:07.097 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-02-05T06:28:07.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:07 smithi142 ceph-mon[30053]: Deploying daemon alertmanager.a on smithi122 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:07 smithi122 ceph-mon[25512]: Deploying daemon alertmanager.a on smithi122 2024-02-05T06:28:07.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:07.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:07 smithi196 ceph-mon[30516]: Deploying daemon alertmanager.a on smithi122 2024-02-05T06:28:07.803 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-02-05T06:28:06.858693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96640,"kb_used_data":3108,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126528,"statfs":{"total":287972524032,"available":287873564672,"internally_reserved":0,"allocated":3182592,"data_stored":2109330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.997785"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849327+0000","last_change":"2024-02-05T06:27:32.440996+0000","last_active":"2024-02-05T06:27:54.849327+0000","last_peered":"2024-02-05T06:27:54.849327+0000","last_clean":"2024-02-05T06:27:54.849327+0000","last_became_active":"2024-02-05T06:27:32.440036+0000","last_became_peered":"2024-02-05T06:27:32.440036+0000","last_unstale":"2024-02-05T06:27:54.849327+0000","last_undegraded":"2024-02-05T06:27:54.849327+0000","last_fullsized":"2024-02-05T06:27:54.849327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:44:43.677631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848082+0000","last_change":"2024-02-05T06:27:30.428443+0000","last_active":"2024-02-05T06:27:54.848082+0000","last_peered":"2024-02-05T06:27:54.848082+0000","last_clean":"2024-02-05T06:27:54.848082+0000","last_became_active":"2024-02-05T06:27:30.428077+0000","last_became_peered":"2024-02-05T06:27:30.428077+0000","last_unstale":"2024-02-05T06:27:54.848082+0000","last_undegraded":"2024-02-05T06:27:54.848082+0000","last_fullsized":"2024-02-05T06:27:54.848082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:07:30.992690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850398+0000","last_change":"2024-02-05T06:27:26.396286+0000","last_active":"2024-02-05T06:27:54.850398+0000","last_peered":"2024-02-05T06:27:54.850398+0000","last_clean":"2024-02-05T06:27:54.850398+0000","last_became_active":"2024-02-05T06:27:26.394815+0000","last_became_peered":"2024-02-05T06:27:26.394815+0000","last_unstale":"2024-02-05T06:27:54.850398+0000","last_undegraded":"2024-02-05T06:27:54.850398+0000","last_fullsized":"2024-02-05T06:27:54.850398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:01:07.273460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848756+0000","last_change":"2024-02-05T06:27:28.407817+0000","last_active":"2024-02-05T06:27:54.848756+0000","last_peered":"2024-02-05T06:27:54.848756+0000","last_clean":"2024-02-05T06:27:54.848756+0000","last_became_active":"2024-02-05T06:27:28.407415+0000","last_became_peered":"2024-02-05T06:27:28.407415+0000","last_unstale":"2024-02-05T06:27:54.848756+0000","last_undegraded":"2024-02-05T06:27:54.848756+0000","last_fullsized":"2024-02-05T06:27:54.848756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:33:08.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847325+0000","last_change":"2024-02-05T06:27:32.446292+0000","last_active":"2024-02-05T06:27:54.847325+0000","last_peered":"2024-02-05T06:27:54.847325+0000","last_clean":"2024-02-05T06:27:54.847325+0000","last_became_active":"2024-02-05T06:27:32.445995+0000","last_became_peered":"2024-02-05T06:27:32.445995+0000","last_unstale":"2024-02-05T06:27:54.847325+0000","last_undegraded":"2024-02-05T06:27:54.847325+0000","last_fullsized":"2024-02-05T06:27:54.847325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:57:33.643235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849020+0000","last_change":"2024-02-05T06:27:30.426001+0000","last_active":"2024-02-05T06:27:54.849020+0000","last_peered":"2024-02-05T06:27:54.849020+0000","last_clean":"2024-02-05T06:27:54.849020+0000","last_became_active":"2024-02-05T06:27:30.424139+0000","last_became_peered":"2024-02-05T06:27:30.424139+0000","last_unstale":"2024-02-05T06:27:54.849020+0000","last_undegraded":"2024-02-05T06:27:54.849020+0000","last_fullsized":"2024-02-05T06:27:54.849020+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:35:29.851130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849872+0000","last_change":"2024-02-05T06:27:26.394370+0000","last_active":"2024-02-05T06:27:54.849872+0000","last_peered":"2024-02-05T06:27:54.849872+0000","last_clean":"2024-02-05T06:27:54.849872+0000","last_became_active":"2024-02-05T06:27:26.393417+0000","last_became_peered":"2024-02-05T06:27:26.393417+0000","last_unstale":"2024-02-05T06:27:54.849872+0000","last_undegraded":"2024-02-05T06:27:54.849872+0000","last_fullsized":"2024-02-05T06:27:54.849872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:56:00.610220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850205+0000","last_change":"2024-02-05T06:27:28.409096+0000","last_active":"2024-02-05T06:27:54.850205+0000","last_peered":"2024-02-05T06:27:54.850205+0000","last_clean":"2024-02-05T06:27:54.850205+0000","last_became_active":"2024-02-05T06:27:28.407786+0000","last_became_peered":"2024-02-05T06:27:28.407786+0000","last_unstale":"2024-02-05T06:27:54.850205+0000","last_undegraded":"2024-02-05T06:27:54.850205+0000","last_fullsized":"2024-02-05T06:27:54.850205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:42:59.049933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848201+0000","last_change":"2024-02-05T06:27:32.445006+0000","last_active":"2024-02-05T06:27:54.848201+0000","last_peered":"2024-02-05T06:27:54.848201+0000","last_clean":"2024-02-05T06:27:54.848201+0000","last_became_active":"2024-02-05T06:27:32.444708+0000","last_became_peered":"2024-02-05T06:27:32.444708+0000","last_unstale":"2024-02-05T06:27:54.848201+0000","last_undegraded":"2024-02-05T06:27:54.848201+0000","last_fullsized":"2024-02-05T06:27:54.848201+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:22:52.917390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849870+0000","last_change":"2024-02-05T06:27:30.429706+0000","last_active":"2024-02-05T06:27:54.849870+0000","last_peered":"2024-02-05T06:27:54.849870+0000","last_clean":"2024-02-05T06:27:54.849870+0000","last_became_active":"2024-02-05T06:27:30.428198+0000","last_became_peered":"2024-02-05T06:27:30.428198+0000","last_unstale":"2024-02-05T06:27:54.849870+0000","last_undegraded":"2024-02-05T06:27:54.849870+0000","last_fullsized":"2024-02-05T06:27:54.849870+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:39:10.169139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851469+0000","last_change":"2024-02-05T06:27:26.397872+0000","last_active":"2024-02-05T06:27:54.851469+0000","last_peered":"2024-02-05T06:27:54.851469+0000","last_clean":"2024-02-05T06:27:54.851469+0000","last_became_active":"2024-02-05T06:27:26.397562+0000","last_became_peered":"2024-02-05T06:27:26.397562+0000","last_unstale":"2024-02-05T06:27:54.851469+0000","last_undegraded":"2024-02-05T06:27:54.851469+0000","last_fullsized":"2024-02-05T06:27:54.851469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:03:03.690389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851811+0000","last_change":"2024-02-05T06:27:28.410333+0000","last_active":"2024-02-05T06:27:54.851811+0000","last_peered":"2024-02-05T06:27:54.851811+0000","last_clean":"2024-02-05T06:27:54.851811+0000","last_became_active":"2024-02-05T06:27:28.409958+0000","last_became_peered":"2024-02-05T06:27:28.409958+0000","last_unstale":"2024-02-05T06:27:54.851811+0000","last_undegraded":"2024-02-05T06:27:54.851811+0000","last_fullsized":"2024-02-05T06:27:54.851811+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:53:32.103202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849745+0000","last_change":"2024-02-05T06:27:32.445983+0000","last_active":"2024-02-05T06:27:54.849745+0000","last_peered":"2024-02-05T06:27:54.849745+0000","last_clean":"2024-02-05T06:27:54.849745+0000","last_became_active":"2024-02-05T06:27:32.445609+0000","last_became_peered":"2024-02-05T06:27:32.445609+0000","last_unstale":"2024-02-05T06:27:54.849745+0000","last_undegraded":"2024-02-05T06:27:54.849745+0000","last_fullsized":"2024-02-05T06:27:54.849745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:17:30.014715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849140+0000","last_change":"2024-02-05T06:27:30.421761+0000","last_active":"2024-02-05T06:27:54.849140+0000","last_peered":"2024-02-05T06:27:54.849140+0000","last_clean":"2024-02-05T06:27:54.849140+0000","last_became_active":"2024-02-05T06:27:30.421177+0000","last_became_peered":"2024-02-05T06:27:30.421177+0000","last_unstale":"2024-02-05T06:27:54.849140+0000","last_undegraded":"2024-02-05T06:27:54.849140+0000","last_fullsized":"2024-02-05T06:27:54.849140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:37:08.277435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849599+0000","last_change":"2024-02-05T06:27:26.398670+0000","last_active":"2024-02-05T06:27:54.849599+0000","last_peered":"2024-02-05T06:27:54.849599+0000","last_clean":"2024-02-05T06:27:54.849599+0000","last_became_active":"2024-02-05T06:27:26.398148+0000","last_became_peered":"2024-02-05T06:27:26.398148+0000","last_unstale":"2024-02-05T06:27:54.849599+0000","last_undegraded":"2024-02-05T06:27:54.849599+0000","last_fullsized":"2024-02-05T06:27:54.849599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:17:55.417724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851381+0000","last_change":"2024-02-05T06:27:28.413706+0000","last_active":"2024-02-05T06:27:54.851381+0000","last_peered":"2024-02-05T06:27:54.851381+0000","last_clean":"2024-02-05T06:27:54.851381+0000","last_became_active":"2024-02-05T06:27:28.413431+0000","last_became_peered":"2024-02-05T06:27:28.413431+0000","last_unstale":"2024-02-05T06:27:54.851381+0000","last_undegraded":"2024-02-05T06:27:54.851381+0000","last_fullsized":"2024-02-05T06:27:54.851381+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:34:22.164454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850149+0000","last_change":"2024-02-05T06:27:32.442862+0000","last_active":"2024-02-05T06:27:54.850149+0000","last_peered":"2024-02-05T06:27:54.850149+0000","last_clean":"2024-02-05T06:27:54.850149+0000","last_became_active":"2024-02-05T06:27:32.441713+0000","last_became_peered":"2024-02-05T06:27:32.441713+0000","last_unstale":"2024-02-05T06:27:54.850149+0000","last_undegraded":"2024-02-05T06:27:54.850149+0000","last_fullsized":"2024-02-05T06:27:54.850149+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:22:41.483099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848933+0000","last_change":"2024-02-05T06:27:30.428018+0000","last_active":"2024-02-05T06:27:54.848933+0000","last_peered":"2024-02-05T06:27:54.848933+0000","last_clean":"2024-02-05T06:27:54.848933+0000","last_became_active":"2024-02-05T06:27:30.427656+0000","last_became_peered":"2024-02-05T06:27:30.427656+0000","last_unstale":"2024-02-05T06:27:54.848933+0000","last_undegraded":"2024-02-05T06:27:54.848933+0000","last_fullsized":"2024-02-05T06:27:54.848933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:05:58.607809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849599+0000","last_change":"2024-02-05T06:27:26.398312+0000","last_active":"2024-02-05T06:27:54.849599+0000","last_peered":"2024-02-05T06:27:54.849599+0000","last_clean":"2024-02-05T06:27:54.849599+0000","last_became_active":"2024-02-05T06:27:26.397665+0000","last_became_peered":"2024-02-05T06:27:26.397665+0000","last_unstale":"2024-02-05T06:27:54.849599+0000","last_undegraded":"2024-02-05T06:27:54.849599+0000","last_fullsized":"2024-02-05T06:27:54.849599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:07:06.141059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849164+0000","last_change":"2024-02-05T06:27:28.412998+0000","last_active":"2024-02-05T06:27:54.849164+0000","last_peered":"2024-02-05T06:27:54.849164+0000","last_clean":"2024-02-05T06:27:54.849164+0000","last_became_active":"2024-02-05T06:27:28.412681+0000","last_became_peered":"2024-02-05T06:27:28.412681+0000","last_unstale":"2024-02-05T06:27:54.849164+0000","last_undegraded":"2024-02-05T06:27:54.849164+0000","last_fullsized":"2024-02-05T06:27:54.849164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:07:13.536299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848496+0000","last_change":"2024-02-05T06:27:32.447131+0000","last_active":"2024-02-05T06:27:54.848496+0000","last_peered":"2024-02-05T06:27:54.848496+0000","last_clean":"2024-02-05T06:27:54.848496+0000","last_became_active":"2024-02-05T06:27:32.446666+0000","last_became_peered":"2024-02-05T06:27:32.446666+0000","last_unstale":"2024-02-05T06:27:54.848496+0000","last_undegraded":"2024-02-05T06:27:54.848496+0000","last_fullsized":"2024-02-05T06:27:54.848496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:35:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849686+0000","last_change":"2024-02-05T06:27:30.427419+0000","last_active":"2024-02-05T06:27:54.849686+0000","last_peered":"2024-02-05T06:27:54.849686+0000","last_clean":"2024-02-05T06:27:54.849686+0000","last_became_active":"2024-02-05T06:27:30.426648+0000","last_became_peered":"2024-02-05T06:27:30.426648+0000","last_unstale":"2024-02-05T06:27:54.849686+0000","last_undegraded":"2024-02-05T06:27:54.849686+0000","last_fullsized":"2024-02-05T06:27:54.849686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:22:24.775157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849512+0000","last_change":"2024-02-05T06:27:26.397528+0000","last_active":"2024-02-05T06:27:54.849512+0000","last_peered":"2024-02-05T06:27:54.849512+0000","last_clean":"2024-02-05T06:27:54.849512+0000","last_became_active":"2024-02-05T06:27:26.396050+0000","last_became_peered":"2024-02-05T06:27:26.396050+0000","last_unstale":"2024-02-05T06:27:54.849512+0000","last_undegraded":"2024-02-05T06:27:54.849512+0000","last_fullsized":"2024-02-05T06:27:54.849512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:07:32.642538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"31'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848998+0000","last_change":"2024-02-05T06:27:28.412601+0000","last_active":"2024-02-05T06:27:54.848998+0000","last_peered":"2024-02-05T06:27:54.848998+0000","last_clean":"2024-02-05T06:27:54.848998+0000","last_became_active":"2024-02-05T06:27:28.411748+0000","last_became_peered":"2024-02-05T06:27:28.411748+0000","last_unstale":"2024-02-05T06:27:54.848998+0000","last_undegraded":"2024-02-05T06:27:54.848998+0000","last_fullsized":"2024-02-05T06:27:54.848998+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:49.872034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848141+0000","last_change":"2024-02-05T06:27:32.448682+0000","last_active":"2024-02-05T06:27:54.848141+0000","last_peered":"2024-02-05T06:27:54.848141+0000","last_clean":"2024-02-05T06:27:54.848141+0000","last_became_active":"2024-02-05T06:27:32.447956+0000","last_became_peered":"2024-02-05T06:27:32.447956+0000","last_unstale":"2024-02-05T06:27:54.848141+0000","last_undegraded":"2024-02-05T06:27:54.848141+0000","last_fullsized":"2024-02-05T06:27:54.848141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:42:50.345955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'8","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444475+0000","last_change":"2024-02-05T06:27:30.428005+0000","last_active":"2024-02-05T06:27:56.444475+0000","last_peered":"2024-02-05T06:27:56.444475+0000","last_clean":"2024-02-05T06:27:56.444475+0000","last_became_active":"2024-02-05T06:27:30.427144+0000","last_became_peered":"2024-02-05T06:27:30.427144+0000","last_unstale":"2024-02-05T06:27:56.444475+0000","last_undegraded":"2024-02-05T06:27:56.444475+0000","last_fullsized":"2024-02-05T06:27:56.444475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:49:49.138800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847845+0000","last_change":"2024-02-05T06:27:26.396069+0000","last_active":"2024-02-05T06:27:54.847845+0000","last_peered":"2024-02-05T06:27:54.847845+0000","last_clean":"2024-02-05T06:27:54.847845+0000","last_became_active":"2024-02-05T06:27:26.395074+0000","last_became_peered":"2024-02-05T06:27:26.395074+0000","last_unstale":"2024-02-05T06:27:54.847845+0000","last_undegraded":"2024-02-05T06:27:54.847845+0000","last_fullsized":"2024-02-05T06:27:54.847845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:32:32.810357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846843+0000","last_change":"2024-02-05T06:27:28.407073+0000","last_active":"2024-02-05T06:27:54.846843+0000","last_peered":"2024-02-05T06:27:54.846843+0000","last_clean":"2024-02-05T06:27:54.846843+0000","last_became_active":"2024-02-05T06:27:28.406720+0000","last_became_peered":"2024-02-05T06:27:28.406720+0000","last_unstale":"2024-02-05T06:27:54.846843+0000","last_undegraded":"2024-02-05T06:27:54.846843+0000","last_fullsized":"2024-02-05T06:27:54.846843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:49:02.062634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850791+0000","last_change":"2024-02-05T06:27:32.446492+0000","last_active":"2024-02-05T06:27:54.850791+0000","last_peered":"2024-02-05T06:27:54.850791+0000","last_clean":"2024-02-05T06:27:54.850791+0000","last_became_active":"2024-02-05T06:27:32.446077+0000","last_became_peered":"2024-02-05T06:27:32.446077+0000","last_unstale":"2024-02-05T06:27:54.850791+0000","last_undegraded":"2024-02-05T06:27:54.850791+0000","last_fullsized":"2024-02-05T06:27:54.850791+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:17:36.979180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848398+0000","last_change":"2024-02-05T06:27:30.430871+0000","last_active":"2024-02-05T06:27:54.848398+0000","last_peered":"2024-02-05T06:27:54.848398+0000","last_clean":"2024-02-05T06:27:54.848398+0000","last_became_active":"2024-02-05T06:27:30.429608+0000","last_became_peered":"2024-02-05T06:27:30.429608+0000","last_unstale":"2024-02-05T06:27:54.848398+0000","last_undegraded":"2024-02-05T06:27:54.848398+0000","last_fullsized":"2024-02-05T06:27:54.848398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:42:58.678081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848340+0000","last_change":"2024-02-05T06:27:26.398260+0000","last_active":"2024-02-05T06:27:54.848340+0000","last_peered":"2024-02-05T06:27:54.848340+0000","last_clean":"2024-02-05T06:27:54.848340+0000","last_became_active":"2024-02-05T06:27:26.397505+0000","last_became_peered":"2024-02-05T06:27:26.397505+0000","last_unstale":"2024-02-05T06:27:54.848340+0000","last_undegraded":"2024-02-05T06:27:54.848340+0000","last_fullsized":"2024-02-05T06:27:54.848340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:48:56.733841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846923+0000","last_change":"2024-02-05T06:27:28.406073+0000","last_active":"2024-02-05T06:27:54.846923+0000","last_peered":"2024-02-05T06:27:54.846923+0000","last_clean":"2024-02-05T06:27:54.846923+0000","last_became_active":"2024-02-05T06:27:28.405310+0000","last_became_peered":"2024-02-05T06:27:28.405310+0000","last_unstale":"2024-02-05T06:27:54.846923+0000","last_undegraded":"2024-02-05T06:27:54.846923+0000","last_fullsized":"2024-02-05T06:27:54.846923+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:21:47.393186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849408+0000","last_change":"2024-02-05T06:27:32.443085+0000","last_active":"2024-02-05T06:27:54.849408+0000","last_peered":"2024-02-05T06:27:54.849408+0000","last_clean":"2024-02-05T06:27:54.849408+0000","last_became_active":"2024-02-05T06:27:32.442638+0000","last_became_peered":"2024-02-05T06:27:32.442638+0000","last_unstale":"2024-02-05T06:27:54.849408+0000","last_undegraded":"2024-02-05T06:27:54.849408+0000","last_fullsized":"2024-02-05T06:27:54.849408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:05:56.215799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846818+0000","last_change":"2024-02-05T06:27:30.428791+0000","last_active":"2024-02-05T06:27:54.846818+0000","last_peered":"2024-02-05T06:27:54.846818+0000","last_clean":"2024-02-05T06:27:54.846818+0000","last_became_active":"2024-02-05T06:27:30.428397+0000","last_became_peered":"2024-02-05T06:27:30.428397+0000","last_unstale":"2024-02-05T06:27:54.846818+0000","last_undegraded":"2024-02-05T06:27:54.846818+0000","last_fullsized":"2024-02-05T06:27:54.846818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:41:57.810713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851952+0000","last_change":"2024-02-05T06:27:26.397397+0000","last_active":"2024-02-05T06:27:54.851952+0000","last_peered":"2024-02-05T06:27:54.851952+0000","last_clean":"2024-02-05T06:27:54.851952+0000","last_became_active":"2024-02-05T06:27:26.397022+0000","last_became_peered":"2024-02-05T06:27:26.397022+0000","last_unstale":"2024-02-05T06:27:54.851952+0000","last_undegraded":"2024-02-05T06:27:54.851952+0000","last_fullsized":"2024-02-05T06:27:54.851952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:46:38.131979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851836+0000","last_change":"2024-02-05T06:27:28.409173+0000","last_active":"2024-02-05T06:27:54.851836+0000","last_peered":"2024-02-05T06:27:54.851836+0000","last_clean":"2024-02-05T06:27:54.851836+0000","last_became_active":"2024-02-05T06:27:28.408804+0000","last_became_peered":"2024-02-05T06:27:28.408804+0000","last_unstale":"2024-02-05T06:27:54.851836+0000","last_undegraded":"2024-02-05T06:27:54.851836+0000","last_fullsized":"2024-02-05T06:27:54.851836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:28:37.097941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849159+0000","last_change":"2024-02-05T06:27:32.442743+0000","last_active":"2024-02-05T06:27:54.849159+0000","last_peered":"2024-02-05T06:27:54.849159+0000","last_clean":"2024-02-05T06:27:54.849159+0000","last_became_active":"2024-02-05T06:27:32.441600+0000","last_became_peered":"2024-02-05T06:27:32.441600+0000","last_unstale":"2024-02-05T06:27:54.849159+0000","last_undegraded":"2024-02-05T06:27:54.849159+0000","last_fullsized":"2024-02-05T06:27:54.849159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:22:12.036136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849958+0000","last_change":"2024-02-05T06:27:30.423661+0000","last_active":"2024-02-05T06:27:54.849958+0000","last_peered":"2024-02-05T06:27:54.849958+0000","last_clean":"2024-02-05T06:27:54.849958+0000","last_became_active":"2024-02-05T06:27:30.423301+0000","last_became_peered":"2024-02-05T06:27:30.423301+0000","last_unstale":"2024-02-05T06:27:54.849958+0000","last_undegraded":"2024-02-05T06:27:54.849958+0000","last_fullsized":"2024-02-05T06:27:54.849958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:43:11.421123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851103+0000","last_change":"2024-02-05T06:27:26.399601+0000","last_active":"2024-02-05T06:27:54.851103+0000","last_peered":"2024-02-05T06:27:54.851103+0000","last_clean":"2024-02-05T06:27:54.851103+0000","last_became_active":"2024-02-05T06:27:26.399269+0000","last_became_peered":"2024-02-05T06:27:26.399269+0000","last_unstale":"2024-02-05T06:27:54.851103+0000","last_undegraded":"2024-02-05T06:27:54.851103+0000","last_fullsized":"2024-02-05T06:27:54.851103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:33:22.859827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"31'6","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850540+0000","last_change":"2024-02-05T06:27:28.407104+0000","last_active":"2024-02-05T06:27:54.850540+0000","last_peered":"2024-02-05T06:27:54.850540+0000","last_clean":"2024-02-05T06:27:54.850540+0000","last_became_active":"2024-02-05T06:27:28.406801+0000","last_became_peered":"2024-02-05T06:27:28.406801+0000","last_unstale":"2024-02-05T06:27:54.850540+0000","last_undegraded":"2024-02-05T06:27:54.850540+0000","last_fullsized":"2024-02-05T06:27:54.850540+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:36:38.771294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848350+0000","last_change":"2024-02-05T06:27:32.444736+0000","last_active":"2024-02-05T06:27:54.848350+0000","last_peered":"2024-02-05T06:27:54.848350+0000","last_clean":"2024-02-05T06:27:54.848350+0000","last_became_active":"2024-02-05T06:27:32.444404+0000","last_became_peered":"2024-02-05T06:27:32.444404+0000","last_unstale":"2024-02-05T06:27:54.848350+0000","last_undegraded":"2024-02-05T06:27:54.848350+0000","last_fullsized":"2024-02-05T06:27:54.848350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:36:14.991005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849976+0000","last_change":"2024-02-05T06:27:30.429431+0000","last_active":"2024-02-05T06:27:54.849976+0000","last_peered":"2024-02-05T06:27:54.849976+0000","last_clean":"2024-02-05T06:27:54.849976+0000","last_became_active":"2024-02-05T06:27:30.427892+0000","last_became_peered":"2024-02-05T06:27:30.427892+0000","last_unstale":"2024-02-05T06:27:54.849976+0000","last_undegraded":"2024-02-05T06:27:54.849976+0000","last_fullsized":"2024-02-05T06:27:54.849976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:39:24.547014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851323+0000","last_change":"2024-02-05T06:27:26.398124+0000","last_active":"2024-02-05T06:27:54.851323+0000","last_peered":"2024-02-05T06:27:54.851323+0000","last_clean":"2024-02-05T06:27:54.851323+0000","last_became_active":"2024-02-05T06:27:26.397672+0000","last_became_peered":"2024-02-05T06:27:26.397672+0000","last_unstale":"2024-02-05T06:27:54.851323+0000","last_undegraded":"2024-02-05T06:27:54.851323+0000","last_fullsized":"2024-02-05T06:27:54.851323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:41:32.865679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851218+0000","last_change":"2024-02-05T06:27:28.410794+0000","last_active":"2024-02-05T06:27:54.851218+0000","last_peered":"2024-02-05T06:27:54.851218+0000","last_clean":"2024-02-05T06:27:54.851218+0000","last_became_active":"2024-02-05T06:27:28.410427+0000","last_became_peered":"2024-02-05T06:27:28.410427+0000","last_unstale":"2024-02-05T06:27:54.851218+0000","last_undegraded":"2024-02-05T06:27:54.851218+0000","last_fullsized":"2024-02-05T06:27:54.851218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:47:12.935091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848549+0000","last_change":"2024-02-05T06:27:32.445626+0000","last_active":"2024-02-05T06:27:54.848549+0000","last_peered":"2024-02-05T06:27:54.848549+0000","last_clean":"2024-02-05T06:27:54.848549+0000","last_became_active":"2024-02-05T06:27:32.445289+0000","last_became_peered":"2024-02-05T06:27:32.445289+0000","last_unstale":"2024-02-05T06:27:54.848549+0000","last_undegraded":"2024-02-05T06:27:54.848549+0000","last_fullsized":"2024-02-05T06:27:54.848549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:28:04.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847367+0000","last_change":"2024-02-05T06:27:30.429732+0000","last_active":"2024-02-05T06:27:54.847367+0000","last_peered":"2024-02-05T06:27:54.847367+0000","last_clean":"2024-02-05T06:27:54.847367+0000","last_became_active":"2024-02-05T06:27:30.429211+0000","last_became_peered":"2024-02-05T06:27:30.429211+0000","last_unstale":"2024-02-05T06:27:54.847367+0000","last_undegraded":"2024-02-05T06:27:54.847367+0000","last_fullsized":"2024-02-05T06:27:54.847367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:27:23.179396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850452+0000","last_change":"2024-02-05T06:27:26.399156+0000","last_active":"2024-02-05T06:27:54.850452+0000","last_peered":"2024-02-05T06:27:54.850452+0000","last_clean":"2024-02-05T06:27:54.850452+0000","last_became_active":"2024-02-05T06:27:26.398705+0000","last_became_peered":"2024-02-05T06:27:26.398705+0000","last_unstale":"2024-02-05T06:27:54.850452+0000","last_undegraded":"2024-02-05T06:27:54.850452+0000","last_fullsized":"2024-02-05T06:27:54.850452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:30:36.542735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849984+0000","last_change":"2024-02-05T06:27:28.413480+0000","last_active":"2024-02-05T06:27:54.849984+0000","last_peered":"2024-02-05T06:27:54.849984+0000","last_clean":"2024-02-05T06:27:54.849984+0000","last_became_active":"2024-02-05T06:27:28.413206+0000","last_became_peered":"2024-02-05T06:27:28.413206+0000","last_unstale":"2024-02-05T06:27:54.849984+0000","last_undegraded":"2024-02-05T06:27:54.849984+0000","last_fullsized":"2024-02-05T06:27:54.849984+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:29:51.921616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847255+0000","last_change":"2024-02-05T06:27:32.448341+0000","last_active":"2024-02-05T06:27:54.847255+0000","last_peered":"2024-02-05T06:27:54.847255+0000","last_clean":"2024-02-05T06:27:54.847255+0000","last_became_active":"2024-02-05T06:27:32.448001+0000","last_became_peered":"2024-02-05T06:27:32.448001+0000","last_unstale":"2024-02-05T06:27:54.847255+0000","last_undegraded":"2024-02-05T06:27:54.847255+0000","last_fullsized":"2024-02-05T06:27:54.847255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:33:22.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444472+0000","last_change":"2024-02-05T06:27:30.427052+0000","last_active":"2024-02-05T06:27:56.444472+0000","last_peered":"2024-02-05T06:27:56.444472+0000","last_clean":"2024-02-05T06:27:56.444472+0000","last_became_active":"2024-02-05T06:27:30.426680+0000","last_became_peered":"2024-02-05T06:27:30.426680+0000","last_unstale":"2024-02-05T06:27:56.444472+0000","last_undegraded":"2024-02-05T06:27:56.444472+0000","last_fullsized":"2024-02-05T06:27:56.444472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:22:15.771257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848799+0000","last_change":"2024-02-05T06:27:26.400082+0000","last_active":"2024-02-05T06:27:54.848799+0000","last_peered":"2024-02-05T06:27:54.848799+0000","last_clean":"2024-02-05T06:27:54.848799+0000","last_became_active":"2024-02-05T06:27:26.399630+0000","last_became_peered":"2024-02-05T06:27:26.399630+0000","last_unstale":"2024-02-05T06:27:54.848799+0000","last_undegraded":"2024-02-05T06:27:54.848799+0000","last_fullsized":"2024-02-05T06:27:54.848799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:32:33.269707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847725+0000","last_change":"2024-02-05T06:27:28.407218+0000","last_active":"2024-02-05T06:27:54.847725+0000","last_peered":"2024-02-05T06:27:54.847725+0000","last_clean":"2024-02-05T06:27:54.847725+0000","last_became_active":"2024-02-05T06:27:28.405368+0000","last_became_peered":"2024-02-05T06:27:28.405368+0000","last_unstale":"2024-02-05T06:27:54.847725+0000","last_undegraded":"2024-02-05T06:27:54.847725+0000","last_fullsized":"2024-02-05T06:27:54.847725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:13:45.383844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850113+0000","last_change":"2024-02-05T06:27:32.442835+0000","last_active":"2024-02-05T06:27:54.850113+0000","last_peered":"2024-02-05T06:27:54.850113+0000","last_clean":"2024-02-05T06:27:54.850113+0000","last_became_active":"2024-02-05T06:27:32.441343+0000","last_became_peered":"2024-02-05T06:27:32.441343+0000","last_unstale":"2024-02-05T06:27:54.850113+0000","last_undegraded":"2024-02-05T06:27:54.850113+0000","last_fullsized":"2024-02-05T06:27:54.850113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:50:11.176760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444492+0000","last_change":"2024-02-05T06:27:30.428355+0000","last_active":"2024-02-05T06:27:56.444492+0000","last_peered":"2024-02-05T06:27:56.444492+0000","last_clean":"2024-02-05T06:27:56.444492+0000","last_became_active":"2024-02-05T06:27:30.428000+0000","last_became_peered":"2024-02-05T06:27:30.428000+0000","last_unstale":"2024-02-05T06:27:56.444492+0000","last_undegraded":"2024-02-05T06:27:56.444492+0000","last_fullsized":"2024-02-05T06:27:56.444492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:41:10.850477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848455+0000","last_change":"2024-02-05T06:27:26.401154+0000","last_active":"2024-02-05T06:27:54.848455+0000","last_peered":"2024-02-05T06:27:54.848455+0000","last_clean":"2024-02-05T06:27:54.848455+0000","last_became_active":"2024-02-05T06:27:26.400337+0000","last_became_peered":"2024-02-05T06:27:26.400337+0000","last_unstale":"2024-02-05T06:27:54.848455+0000","last_undegraded":"2024-02-05T06:27:54.848455+0000","last_fullsized":"2024-02-05T06:27:54.848455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:50:16.332499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850342+0000","last_change":"2024-02-05T06:27:28.408308+0000","last_active":"2024-02-05T06:27:54.850342+0000","last_peered":"2024-02-05T06:27:54.850342+0000","last_clean":"2024-02-05T06:27:54.850342+0000","last_became_active":"2024-02-05T06:27:28.407935+0000","last_became_peered":"2024-02-05T06:27:28.407935+0000","last_unstale":"2024-02-05T06:27:54.850342+0000","last_undegraded":"2024-02-05T06:27:54.850342+0000","last_fullsized":"2024-02-05T06:27:54.850342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:45:56.352527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850593+0000","last_change":"2024-02-05T06:27:32.442800+0000","last_active":"2024-02-05T06:27:54.850593+0000","last_peered":"2024-02-05T06:27:54.850593+0000","last_clean":"2024-02-05T06:27:54.850593+0000","last_became_active":"2024-02-05T06:27:32.441702+0000","last_became_peered":"2024-02-05T06:27:32.441702+0000","last_unstale":"2024-02-05T06:27:54.850593+0000","last_undegraded":"2024-02-05T06:27:54.850593+0000","last_fullsized":"2024-02-05T06:27:54.850593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:13:02.611403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850046+0000","last_change":"2024-02-05T06:27:30.430589+0000","last_active":"2024-02-05T06:27:54.850046+0000","last_peered":"2024-02-05T06:27:54.850046+0000","last_clean":"2024-02-05T06:27:54.850046+0000","last_became_active":"2024-02-05T06:27:30.429718+0000","last_became_peered":"2024-02-05T06:27:30.429718+0000","last_unstale":"2024-02-05T06:27:54.850046+0000","last_undegraded":"2024-02-05T06:27:54.850046+0000","last_fullsized":"2024-02-05T06:27:54.850046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:30:31.957515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849054+0000","last_change":"2024-02-05T06:27:26.395707+0000","last_active":"2024-02-05T06:27:54.849054+0000","last_peered":"2024-02-05T06:27:54.849054+0000","last_clean":"2024-02-05T06:27:54.849054+0000","last_became_active":"2024-02-05T06:27:26.395402+0000","last_became_peered":"2024-02-05T06:27:26.395402+0000","last_unstale":"2024-02-05T06:27:54.849054+0000","last_undegraded":"2024-02-05T06:27:54.849054+0000","last_fullsized":"2024-02-05T06:27:54.849054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:47:08.789979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"31'4","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848659+0000","last_change":"2024-02-05T06:27:28.409566+0000","last_active":"2024-02-05T06:27:54.848659+0000","last_peered":"2024-02-05T06:27:54.848659+0000","last_clean":"2024-02-05T06:27:54.848659+0000","last_became_active":"2024-02-05T06:27:28.408853+0000","last_became_peered":"2024-02-05T06:27:28.408853+0000","last_unstale":"2024-02-05T06:27:54.848659+0000","last_undegraded":"2024-02-05T06:27:54.848659+0000","last_fullsized":"2024-02-05T06:27:54.848659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:35:37.974941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848738+0000","last_change":"2024-02-05T06:27:32.447223+0000","last_active":"2024-02-05T06:27:54.848738+0000","last_peered":"2024-02-05T06:27:54.848738+0000","last_clean":"2024-02-05T06:27:54.848738+0000","last_became_active":"2024-02-05T06:27:32.446866+0000","last_became_peered":"2024-02-05T06:27:32.446866+0000","last_unstale":"2024-02-05T06:27:54.848738+0000","last_undegraded":"2024-02-05T06:27:54.848738+0000","last_fullsized":"2024-02-05T06:27:54.848738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:17:12.028749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848482+0000","last_change":"2024-02-05T06:27:30.429780+0000","last_active":"2024-02-05T06:27:54.848482+0000","last_peered":"2024-02-05T06:27:54.848482+0000","last_clean":"2024-02-05T06:27:54.848482+0000","last_became_active":"2024-02-05T06:27:30.429275+0000","last_became_peered":"2024-02-05T06:27:30.429275+0000","last_unstale":"2024-02-05T06:27:54.848482+0000","last_undegraded":"2024-02-05T06:27:54.848482+0000","last_fullsized":"2024-02-05T06:27:54.848482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:56:40.686628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848501+0000","last_change":"2024-02-05T06:27:26.401129+0000","last_active":"2024-02-05T06:27:54.848501+0000","last_peered":"2024-02-05T06:27:54.848501+0000","last_clean":"2024-02-05T06:27:54.848501+0000","last_became_active":"2024-02-05T06:27:26.400591+0000","last_became_peered":"2024-02-05T06:27:26.400591+0000","last_unstale":"2024-02-05T06:27:54.848501+0000","last_undegraded":"2024-02-05T06:27:54.848501+0000","last_fullsized":"2024-02-05T06:27:54.848501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:56:01.716537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848869+0000","last_change":"2024-02-05T06:27:28.412546+0000","last_active":"2024-02-05T06:27:54.848869+0000","last_peered":"2024-02-05T06:27:54.848869+0000","last_clean":"2024-02-05T06:27:54.848869+0000","last_became_active":"2024-02-05T06:27:28.412211+0000","last_became_peered":"2024-02-05T06:27:28.412211+0000","last_unstale":"2024-02-05T06:27:54.848869+0000","last_undegraded":"2024-02-05T06:27:54.848869+0000","last_fullsized":"2024-02-05T06:27:54.848869+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:54:51.755993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849828+0000","last_change":"2024-02-05T06:27:32.449484+0000","last_active":"2024-02-05T06:27:54.849828+0000","last_peered":"2024-02-05T06:27:54.849828+0000","last_clean":"2024-02-05T06:27:54.849828+0000","last_became_active":"2024-02-05T06:27:32.449190+0000","last_became_peered":"2024-02-05T06:27:32.449190+0000","last_unstale":"2024-02-05T06:27:54.849828+0000","last_undegraded":"2024-02-05T06:27:54.849828+0000","last_fullsized":"2024-02-05T06:27:54.849828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:19:12.198205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851223+0000","last_change":"2024-02-05T06:27:26.399240+0000","last_active":"2024-02-05T06:27:54.851223+0000","last_peered":"2024-02-05T06:27:54.851223+0000","last_clean":"2024-02-05T06:27:54.851223+0000","last_became_active":"2024-02-05T06:27:26.398837+0000","last_became_peered":"2024-02-05T06:27:26.398837+0000","last_unstale":"2024-02-05T06:27:54.851223+0000","last_undegraded":"2024-02-05T06:27:54.851223+0000","last_fullsized":"2024-02-05T06:27:54.851223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:13:16.694898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849932+0000","last_change":"2024-02-05T06:27:28.413614+0000","last_active":"2024-02-05T06:27:54.849932+0000","last_peered":"2024-02-05T06:27:54.849932+0000","last_clean":"2024-02-05T06:27:54.849932+0000","last_became_active":"2024-02-05T06:27:28.413323+0000","last_became_peered":"2024-02-05T06:27:28.413323+0000","last_unstale":"2024-02-05T06:27:54.849932+0000","last_undegraded":"2024-02-05T06:27:54.849932+0000","last_fullsized":"2024-02-05T06:27:54.849932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:43:59.406472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'8","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444533+0000","last_change":"2024-02-05T06:27:30.427941+0000","last_active":"2024-02-05T06:27:56.444533+0000","last_peered":"2024-02-05T06:27:56.444533+0000","last_clean":"2024-02-05T06:27:56.444533+0000","last_became_active":"2024-02-05T06:27:30.427210+0000","last_became_peered":"2024-02-05T06:27:30.427210+0000","last_unstale":"2024-02-05T06:27:56.444533+0000","last_undegraded":"2024-02-05T06:27:56.444533+0000","last_fullsized":"2024-02-05T06:27:56.444533+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:06:14.803220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847183+0000","last_change":"2024-02-05T06:27:32.449362+0000","last_active":"2024-02-05T06:27:54.847183+0000","last_peered":"2024-02-05T06:27:54.847183+0000","last_clean":"2024-02-05T06:27:54.847183+0000","last_became_active":"2024-02-05T06:27:32.449094+0000","last_became_peered":"2024-02-05T06:27:32.449094+0000","last_unstale":"2024-02-05T06:27:54.847183+0000","last_undegraded":"2024-02-05T06:27:54.847183+0000","last_fullsized":"2024-02-05T06:27:54.847183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:18.263817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848455+0000","last_change":"2024-02-05T06:27:26.393500+0000","last_active":"2024-02-05T06:27:54.848455+0000","last_peered":"2024-02-05T06:27:54.848455+0000","last_clean":"2024-02-05T06:27:54.848455+0000","last_became_active":"2024-02-05T06:27:26.392118+0000","last_became_peered":"2024-02-05T06:27:26.392118+0000","last_unstale":"2024-02-05T06:27:54.848455+0000","last_undegraded":"2024-02-05T06:27:54.848455+0000","last_fullsized":"2024-02-05T06:27:54.848455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:04:52.273437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"31'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444592+0000","last_change":"2024-02-05T06:27:28.409328+0000","last_active":"2024-02-05T06:27:56.444592+0000","last_peered":"2024-02-05T06:27:56.444592+0000","last_clean":"2024-02-05T06:27:56.444592+0000","last_became_active":"2024-02-05T06:27:28.409003+0000","last_became_peered":"2024-02-05T06:27:28.409003+0000","last_unstale":"2024-02-05T06:27:56.444592+0000","last_undegraded":"2024-02-05T06:27:56.444592+0000","last_fullsized":"2024-02-05T06:27:56.444592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:42:32.044272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848371+0000","last_change":"2024-02-05T06:27:30.423065+0000","last_active":"2024-02-05T06:27:54.848371+0000","last_peered":"2024-02-05T06:27:54.848371+0000","last_clean":"2024-02-05T06:27:54.848371+0000","last_became_active":"2024-02-05T06:27:30.422780+0000","last_became_peered":"2024-02-05T06:27:30.422780+0000","last_unstale":"2024-02-05T06:27:54.848371+0000","last_undegraded":"2024-02-05T06:27:54.848371+0000","last_fullsized":"2024-02-05T06:27:54.848371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:45:46.674581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847701+0000","last_change":"2024-02-05T06:27:32.447396+0000","last_active":"2024-02-05T06:27:54.847701+0000","last_peered":"2024-02-05T06:27:54.847701+0000","last_clean":"2024-02-05T06:27:54.847701+0000","last_became_active":"2024-02-05T06:27:32.446490+0000","last_became_peered":"2024-02-05T06:27:32.446490+0000","last_unstale":"2024-02-05T06:27:54.847701+0000","last_undegraded":"2024-02-05T06:27:54.847701+0000","last_fullsized":"2024-02-05T06:27:54.847701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:15:28.025147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849410+0000","last_change":"2024-02-05T06:27:26.397713+0000","last_active":"2024-02-05T06:27:54.849410+0000","last_peered":"2024-02-05T06:27:54.849410+0000","last_clean":"2024-02-05T06:27:54.849410+0000","last_became_active":"2024-02-05T06:27:26.396996+0000","last_became_peered":"2024-02-05T06:27:26.396996+0000","last_unstale":"2024-02-05T06:27:54.849410+0000","last_undegraded":"2024-02-05T06:27:54.849410+0000","last_fullsized":"2024-02-05T06:27:54.849410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:54:35.310922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'19","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850514+0000","last_change":"2024-02-05T06:27:28.410679+0000","last_active":"2024-02-05T06:27:54.850514+0000","last_peered":"2024-02-05T06:27:54.850514+0000","last_clean":"2024-02-05T06:27:54.850514+0000","last_became_active":"2024-02-05T06:27:28.410179+0000","last_became_peered":"2024-02-05T06:27:28.410179+0000","last_unstale":"2024-02-05T06:27:54.850514+0000","last_undegraded":"2024-02-05T06:27:54.850514+0000","last_fullsized":"2024-02-05T06:27:54.850514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:07:28.997364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847312+0000","last_change":"2024-02-05T06:27:30.429578+0000","last_active":"2024-02-05T06:27:54.847312+0000","last_peered":"2024-02-05T06:27:54.847312+0000","last_clean":"2024-02-05T06:27:54.847312+0000","last_became_active":"2024-02-05T06:27:30.429157+0000","last_became_peered":"2024-02-05T06:27:30.429157+0000","last_unstale":"2024-02-05T06:27:54.847312+0000","last_undegraded":"2024-02-05T06:27:54.847312+0000","last_fullsized":"2024-02-05T06:27:54.847312+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:02:20.685813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850129+0000","last_change":"2024-02-05T06:27:32.449298+0000","last_active":"2024-02-05T06:27:54.850129+0000","last_peered":"2024-02-05T06:27:54.850129+0000","last_clean":"2024-02-05T06:27:54.850129+0000","last_became_active":"2024-02-05T06:27:32.449025+0000","last_became_peered":"2024-02-05T06:27:32.449025+0000","last_unstale":"2024-02-05T06:27:54.850129+0000","last_undegraded":"2024-02-05T06:27:54.850129+0000","last_fullsized":"2024-02-05T06:27:54.850129+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:08:23.389052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"31'18","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847995+0000","last_change":"2024-02-05T06:27:28.412028+0000","last_active":"2024-02-05T06:27:54.847995+0000","last_peered":"2024-02-05T06:27:54.847995+0000","last_clean":"2024-02-05T06:27:54.847995+0000","last_became_active":"2024-02-05T06:27:28.411744+0000","last_became_peered":"2024-02-05T06:27:28.411744+0000","last_unstale":"2024-02-05T06:27:54.847995+0000","last_undegraded":"2024-02-05T06:27:54.847995+0000","last_fullsized":"2024-02-05T06:27:54.847995+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:18:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847074+0000","last_change":"2024-02-05T06:27:26.396383+0000","last_active":"2024-02-05T06:27:54.847074+0000","last_peered":"2024-02-05T06:27:54.847074+0000","last_clean":"2024-02-05T06:27:54.847074+0000","last_became_active":"2024-02-05T06:27:26.395295+0000","last_became_peered":"2024-02-05T06:27:26.395295+0000","last_unstale":"2024-02-05T06:27:54.847074+0000","last_undegraded":"2024-02-05T06:27:54.847074+0000","last_fullsized":"2024-02-05T06:27:54.847074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:04:52.530255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850482+0000","last_change":"2024-02-05T06:27:30.430640+0000","last_active":"2024-02-05T06:27:54.850482+0000","last_peered":"2024-02-05T06:27:54.850482+0000","last_clean":"2024-02-05T06:27:54.850482+0000","last_became_active":"2024-02-05T06:27:30.430026+0000","last_became_peered":"2024-02-05T06:27:30.430026+0000","last_unstale":"2024-02-05T06:27:54.850482+0000","last_undegraded":"2024-02-05T06:27:54.850482+0000","last_fullsized":"2024-02-05T06:27:54.850482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:22:14.616175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848781+0000","last_change":"2024-02-05T06:27:32.445713+0000","last_active":"2024-02-05T06:27:54.848781+0000","last_peered":"2024-02-05T06:27:54.848781+0000","last_clean":"2024-02-05T06:27:54.848781+0000","last_became_active":"2024-02-05T06:27:32.445215+0000","last_became_peered":"2024-02-05T06:27:32.445215+0000","last_unstale":"2024-02-05T06:27:54.848781+0000","last_undegraded":"2024-02-05T06:27:54.848781+0000","last_fullsized":"2024-02-05T06:27:54.848781+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:32:05.241000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847471+0000","last_change":"2024-02-05T06:27:28.405805+0000","last_active":"2024-02-05T06:27:54.847471+0000","last_peered":"2024-02-05T06:27:54.847471+0000","last_clean":"2024-02-05T06:27:54.847471+0000","last_became_active":"2024-02-05T06:27:28.405476+0000","last_became_peered":"2024-02-05T06:27:28.405476+0000","last_unstale":"2024-02-05T06:27:54.847471+0000","last_undegraded":"2024-02-05T06:27:54.847471+0000","last_fullsized":"2024-02-05T06:27:54.847471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:29:30.336402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850069+0000","last_change":"2024-02-05T06:27:26.394195+0000","last_active":"2024-02-05T06:27:54.850069+0000","last_peered":"2024-02-05T06:27:54.850069+0000","last_clean":"2024-02-05T06:27:54.850069+0000","last_became_active":"2024-02-05T06:27:26.392756+0000","last_became_peered":"2024-02-05T06:27:26.392756+0000","last_unstale":"2024-02-05T06:27:54.850069+0000","last_undegraded":"2024-02-05T06:27:54.850069+0000","last_fullsized":"2024-02-05T06:27:54.850069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:52:15.286462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"32'208","reported_seq":314,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.943168+0000","last_change":"2024-02-05T06:27:16.867788+0000","last_active":"2024-02-05T06:27:56.943168+0000","last_peered":"2024-02-05T06:27:56.943168+0000","last_clean":"2024-02-05T06:27:56.943168+0000","last_became_active":"2024-02-05T06:27:16.866776+0000","last_became_peered":"2024-02-05T06:27:16.866776+0000","last_unstale":"2024-02-05T06:27:56.943168+0000","last_undegraded":"2024-02-05T06:27:56.943168+0000","last_fullsized":"2024-02-05T06:27:56.943168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:15.855371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:15.855371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:15.855371+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:48:36.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850375+0000","last_change":"2024-02-05T06:27:30.422515+0000","last_active":"2024-02-05T06:27:54.850375+0000","last_peered":"2024-02-05T06:27:54.850375+0000","last_clean":"2024-02-05T06:27:54.850375+0000","last_became_active":"2024-02-05T06:27:30.422160+0000","last_became_peered":"2024-02-05T06:27:30.422160+0000","last_unstale":"2024-02-05T06:27:54.850375+0000","last_undegraded":"2024-02-05T06:27:54.850375+0000","last_fullsized":"2024-02-05T06:27:54.850375+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:29:54.751053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850861+0000","last_change":"2024-02-05T06:27:32.446348+0000","last_active":"2024-02-05T06:27:54.850861+0000","last_peered":"2024-02-05T06:27:54.850861+0000","last_clean":"2024-02-05T06:27:54.850861+0000","last_became_active":"2024-02-05T06:27:32.445926+0000","last_became_peered":"2024-02-05T06:27:32.445926+0000","last_unstale":"2024-02-05T06:27:54.850861+0000","last_undegraded":"2024-02-05T06:27:54.850861+0000","last_fullsized":"2024-02-05T06:27:54.850861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:25:37.424551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"31'14","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847943+0000","last_change":"2024-02-05T06:27:28.406627+0000","last_active":"2024-02-05T06:27:54.847943+0000","last_peered":"2024-02-05T06:27:54.847943+0000","last_clean":"2024-02-05T06:27:54.847943+0000","last_became_active":"2024-02-05T06:27:28.405647+0000","last_became_peered":"2024-02-05T06:27:28.405647+0000","last_unstale":"2024-02-05T06:27:54.847943+0000","last_undegraded":"2024-02-05T06:27:54.847943+0000","last_fullsized":"2024-02-05T06:27:54.847943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:50:21.174223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848771+0000","last_change":"2024-02-05T06:27:26.397456+0000","last_active":"2024-02-05T06:27:54.848771+0000","last_peered":"2024-02-05T06:27:54.848771+0000","last_clean":"2024-02-05T06:27:54.848771+0000","last_became_active":"2024-02-05T06:27:26.396961+0000","last_became_peered":"2024-02-05T06:27:26.396961+0000","last_unstale":"2024-02-05T06:27:54.848771+0000","last_undegraded":"2024-02-05T06:27:54.848771+0000","last_fullsized":"2024-02-05T06:27:54.848771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:15:40.059637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849302+0000","last_change":"2024-02-05T06:27:30.427398+0000","last_active":"2024-02-05T06:27:54.849302+0000","last_peered":"2024-02-05T06:27:54.849302+0000","last_clean":"2024-02-05T06:27:54.849302+0000","last_became_active":"2024-02-05T06:27:30.426939+0000","last_became_peered":"2024-02-05T06:27:30.426939+0000","last_unstale":"2024-02-05T06:27:54.849302+0000","last_undegraded":"2024-02-05T06:27:54.849302+0000","last_fullsized":"2024-02-05T06:27:54.849302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:30:56.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849434+0000","last_change":"2024-02-05T06:27:32.441103+0000","last_active":"2024-02-05T06:27:54.849434+0000","last_peered":"2024-02-05T06:27:54.849434+0000","last_clean":"2024-02-05T06:27:54.849434+0000","last_became_active":"2024-02-05T06:27:32.440031+0000","last_became_peered":"2024-02-05T06:27:32.440031+0000","last_unstale":"2024-02-05T06:27:54.849434+0000","last_undegraded":"2024-02-05T06:27:54.849434+0000","last_fullsized":"2024-02-05T06:27:54.849434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:52:59.139877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851222+0000","last_change":"2024-02-05T06:27:26.400277+0000","last_active":"2024-02-05T06:27:54.851222+0000","last_peered":"2024-02-05T06:27:54.851222+0000","last_clean":"2024-02-05T06:27:54.851222+0000","last_became_active":"2024-02-05T06:27:26.399980+0000","last_became_peered":"2024-02-05T06:27:26.399980+0000","last_unstale":"2024-02-05T06:27:54.851222+0000","last_undegraded":"2024-02-05T06:27:54.851222+0000","last_fullsized":"2024-02-05T06:27:54.851222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:09:02.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'28","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444496+0000","last_change":"2024-02-05T06:27:28.408831+0000","last_active":"2024-02-05T06:27:56.444496+0000","last_peered":"2024-02-05T06:27:56.444496+0000","last_clean":"2024-02-05T06:27:56.444496+0000","last_became_active":"2024-02-05T06:27:28.407476+0000","last_became_peered":"2024-02-05T06:27:28.407476+0000","last_unstale":"2024-02-05T06:27:56.444496+0000","last_undegraded":"2024-02-05T06:27:56.444496+0000","last_fullsized":"2024-02-05T06:27:56.444496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:37:13.422228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444488+0000","last_change":"2024-02-05T06:27:30.427766+0000","last_active":"2024-02-05T06:27:56.444488+0000","last_peered":"2024-02-05T06:27:56.444488+0000","last_clean":"2024-02-05T06:27:56.444488+0000","last_became_active":"2024-02-05T06:27:30.426677+0000","last_became_peered":"2024-02-05T06:27:30.426677+0000","last_unstale":"2024-02-05T06:27:56.444488+0000","last_undegraded":"2024-02-05T06:27:56.444488+0000","last_fullsized":"2024-02-05T06:27:56.444488+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:10:23.582817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848673+0000","last_change":"2024-02-05T06:27:32.443000+0000","last_active":"2024-02-05T06:27:54.848673+0000","last_peered":"2024-02-05T06:27:54.848673+0000","last_clean":"2024-02-05T06:27:54.848673+0000","last_became_active":"2024-02-05T06:27:32.441676+0000","last_became_peered":"2024-02-05T06:27:32.441676+0000","last_unstale":"2024-02-05T06:27:54.848673+0000","last_undegraded":"2024-02-05T06:27:54.848673+0000","last_fullsized":"2024-02-05T06:27:54.848673+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:57:40.607628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850023+0000","last_change":"2024-02-05T06:27:26.394055+0000","last_active":"2024-02-05T06:27:54.850023+0000","last_peered":"2024-02-05T06:27:54.850023+0000","last_clean":"2024-02-05T06:27:54.850023+0000","last_became_active":"2024-02-05T06:27:26.392212+0000","last_became_peered":"2024-02-05T06:27:26.392212+0000","last_unstale":"2024-02-05T06:27:54.850023+0000","last_undegraded":"2024-02-05T06:27:54.850023+0000","last_fullsized":"2024-02-05T06:27:54.850023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:22:05.181422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"31'13","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850611+0000","last_change":"2024-02-05T06:27:28.409342+0000","last_active":"2024-02-05T06:27:54.850611+0000","last_peered":"2024-02-05T06:27:54.850611+0000","last_clean":"2024-02-05T06:27:54.850611+0000","last_became_active":"2024-02-05T06:27:28.409034+0000","last_became_peered":"2024-02-05T06:27:28.409034+0000","last_unstale":"2024-02-05T06:27:54.850611+0000","last_undegraded":"2024-02-05T06:27:54.850611+0000","last_fullsized":"2024-02-05T06:27:54.850611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:30:21.100994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444478+0000","last_change":"2024-02-05T06:27:30.431185+0000","last_active":"2024-02-05T06:27:56.444478+0000","last_peered":"2024-02-05T06:27:56.444478+0000","last_clean":"2024-02-05T06:27:56.444478+0000","last_became_active":"2024-02-05T06:27:30.430696+0000","last_became_peered":"2024-02-05T06:27:30.430696+0000","last_unstale":"2024-02-05T06:27:56.444478+0000","last_undegraded":"2024-02-05T06:27:56.444478+0000","last_fullsized":"2024-02-05T06:27:56.444478+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:36:46.849708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849743+0000","last_change":"2024-02-05T06:27:32.445602+0000","last_active":"2024-02-05T06:27:54.849743+0000","last_peered":"2024-02-05T06:27:54.849743+0000","last_clean":"2024-02-05T06:27:54.849743+0000","last_became_active":"2024-02-05T06:27:32.445229+0000","last_became_peered":"2024-02-05T06:27:32.445229+0000","last_unstale":"2024-02-05T06:27:54.849743+0000","last_undegraded":"2024-02-05T06:27:54.849743+0000","last_fullsized":"2024-02-05T06:27:54.849743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:50:15.363905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851721+0000","last_change":"2024-02-05T06:27:26.397751+0000","last_active":"2024-02-05T06:27:54.851721+0000","last_peered":"2024-02-05T06:27:54.851721+0000","last_clean":"2024-02-05T06:27:54.851721+0000","last_became_active":"2024-02-05T06:27:26.397412+0000","last_became_peered":"2024-02-05T06:27:26.397412+0000","last_unstale":"2024-02-05T06:27:54.851721+0000","last_undegraded":"2024-02-05T06:27:54.851721+0000","last_fullsized":"2024-02-05T06:27:54.851721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:12:38.595103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"31'12","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848631+0000","last_change":"2024-02-05T06:27:28.406087+0000","last_active":"2024-02-05T06:27:54.848631+0000","last_peered":"2024-02-05T06:27:54.848631+0000","last_clean":"2024-02-05T06:27:54.848631+0000","last_became_active":"2024-02-05T06:27:28.405781+0000","last_became_peered":"2024-02-05T06:27:28.405781+0000","last_unstale":"2024-02-05T06:27:54.848631+0000","last_undegraded":"2024-02-05T06:27:54.848631+0000","last_fullsized":"2024-02-05T06:27:54.848631+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:31:31.341740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848197+0000","last_change":"2024-02-05T06:27:30.423184+0000","last_active":"2024-02-05T06:27:54.848197+0000","last_peered":"2024-02-05T06:27:54.848197+0000","last_clean":"2024-02-05T06:27:54.848197+0000","last_became_active":"2024-02-05T06:27:30.422888+0000","last_became_peered":"2024-02-05T06:27:30.422888+0000","last_unstale":"2024-02-05T06:27:54.848197+0000","last_undegraded":"2024-02-05T06:27:54.848197+0000","last_fullsized":"2024-02-05T06:27:54.848197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:26:21.047318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851086+0000","last_change":"2024-02-05T06:27:32.446254+0000","last_active":"2024-02-05T06:27:54.851086+0000","last_peered":"2024-02-05T06:27:54.851086+0000","last_clean":"2024-02-05T06:27:54.851086+0000","last_became_active":"2024-02-05T06:27:32.445736+0000","last_became_peered":"2024-02-05T06:27:32.445736+0000","last_unstale":"2024-02-05T06:27:54.851086+0000","last_undegraded":"2024-02-05T06:27:54.851086+0000","last_fullsized":"2024-02-05T06:27:54.851086+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:22:34.004152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444587+0000","last_change":"2024-02-05T06:27:30.422795+0000","last_active":"2024-02-05T06:27:56.444587+0000","last_peered":"2024-02-05T06:27:56.444587+0000","last_clean":"2024-02-05T06:27:56.444587+0000","last_became_active":"2024-02-05T06:27:30.422307+0000","last_became_peered":"2024-02-05T06:27:30.422307+0000","last_unstale":"2024-02-05T06:27:56.444587+0000","last_undegraded":"2024-02-05T06:27:56.444587+0000","last_fullsized":"2024-02-05T06:27:56.444587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:22.186920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848834+0000","last_change":"2024-02-05T06:27:26.399213+0000","last_active":"2024-02-05T06:27:54.848834+0000","last_peered":"2024-02-05T06:27:54.848834+0000","last_clean":"2024-02-05T06:27:54.848834+0000","last_became_active":"2024-02-05T06:27:26.398585+0000","last_became_peered":"2024-02-05T06:27:26.398585+0000","last_unstale":"2024-02-05T06:27:54.848834+0000","last_undegraded":"2024-02-05T06:27:54.848834+0000","last_fullsized":"2024-02-05T06:27:54.848834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:19:26.201195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851047+0000","last_change":"2024-02-05T06:27:28.412084+0000","last_active":"2024-02-05T06:27:54.851047+0000","last_peered":"2024-02-05T06:27:54.851047+0000","last_clean":"2024-02-05T06:27:54.851047+0000","last_became_active":"2024-02-05T06:27:28.411769+0000","last_became_peered":"2024-02-05T06:27:28.411769+0000","last_unstale":"2024-02-05T06:27:54.851047+0000","last_undegraded":"2024-02-05T06:27:54.851047+0000","last_fullsized":"2024-02-05T06:27:54.851047+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:50:47.398036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849673+0000","last_change":"2024-02-05T06:27:32.445521+0000","last_active":"2024-02-05T06:27:54.849673+0000","last_peered":"2024-02-05T06:27:54.849673+0000","last_clean":"2024-02-05T06:27:54.849673+0000","last_became_active":"2024-02-05T06:27:32.445017+0000","last_became_peered":"2024-02-05T06:27:32.445017+0000","last_unstale":"2024-02-05T06:27:54.849673+0000","last_undegraded":"2024-02-05T06:27:54.849673+0000","last_fullsized":"2024-02-05T06:27:54.849673+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:17:00.166203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850146+0000","last_change":"2024-02-05T06:27:30.430250+0000","last_active":"2024-02-05T06:27:54.850146+0000","last_peered":"2024-02-05T06:27:54.850146+0000","last_clean":"2024-02-05T06:27:54.850146+0000","last_became_active":"2024-02-05T06:27:30.429003+0000","last_became_peered":"2024-02-05T06:27:30.429003+0000","last_unstale":"2024-02-05T06:27:54.850146+0000","last_undegraded":"2024-02-05T06:27:54.850146+0000","last_fullsized":"2024-02-05T06:27:54.850146+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:20:11.672144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847425+0000","last_change":"2024-02-05T06:27:26.395355+0000","last_active":"2024-02-05T06:27:54.847425+0000","last_peered":"2024-02-05T06:27:54.847425+0000","last_clean":"2024-02-05T06:27:54.847425+0000","last_became_active":"2024-02-05T06:27:26.394700+0000","last_became_peered":"2024-02-05T06:27:26.394700+0000","last_unstale":"2024-02-05T06:27:54.847425+0000","last_undegraded":"2024-02-05T06:27:54.847425+0000","last_fullsized":"2024-02-05T06:27:54.847425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:14:47.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848617+0000","last_change":"2024-02-05T06:27:28.412358+0000","last_active":"2024-02-05T06:27:54.848617+0000","last_peered":"2024-02-05T06:27:54.848617+0000","last_clean":"2024-02-05T06:27:54.848617+0000","last_became_active":"2024-02-05T06:27:28.411707+0000","last_became_peered":"2024-02-05T06:27:28.411707+0000","last_unstale":"2024-02-05T06:27:54.848617+0000","last_undegraded":"2024-02-05T06:27:54.848617+0000","last_fullsized":"2024-02-05T06:27:54.848617+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:23:02.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848848+0000","last_change":"2024-02-05T06:27:32.446030+0000","last_active":"2024-02-05T06:27:54.848848+0000","last_peered":"2024-02-05T06:27:54.848848+0000","last_clean":"2024-02-05T06:27:54.848848+0000","last_became_active":"2024-02-05T06:27:32.445738+0000","last_became_peered":"2024-02-05T06:27:32.445738+0000","last_unstale":"2024-02-05T06:27:54.848848+0000","last_undegraded":"2024-02-05T06:27:54.848848+0000","last_fullsized":"2024-02-05T06:27:54.848848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:09:14.377854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848971+0000","last_change":"2024-02-05T06:27:26.398020+0000","last_active":"2024-02-05T06:27:54.848971+0000","last_peered":"2024-02-05T06:27:54.848971+0000","last_clean":"2024-02-05T06:27:54.848971+0000","last_became_active":"2024-02-05T06:27:26.397640+0000","last_became_peered":"2024-02-05T06:27:26.397640+0000","last_unstale":"2024-02-05T06:27:54.848971+0000","last_undegraded":"2024-02-05T06:27:54.848971+0000","last_fullsized":"2024-02-05T06:27:54.848971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:26:43.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850137+0000","last_change":"2024-02-05T06:27:28.405872+0000","last_active":"2024-02-05T06:27:54.850137+0000","last_peered":"2024-02-05T06:27:54.850137+0000","last_clean":"2024-02-05T06:27:54.850137+0000","last_became_active":"2024-02-05T06:27:28.405538+0000","last_became_peered":"2024-02-05T06:27:28.405538+0000","last_unstale":"2024-02-05T06:27:54.850137+0000","last_undegraded":"2024-02-05T06:27:54.850137+0000","last_fullsized":"2024-02-05T06:27:54.850137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:15:30.126689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850254+0000","last_change":"2024-02-05T06:27:30.424420+0000","last_active":"2024-02-05T06:27:54.850254+0000","last_peered":"2024-02-05T06:27:54.850254+0000","last_clean":"2024-02-05T06:27:54.850254+0000","last_became_active":"2024-02-05T06:27:30.424016+0000","last_became_peered":"2024-02-05T06:27:30.424016+0000","last_unstale":"2024-02-05T06:27:54.850254+0000","last_undegraded":"2024-02-05T06:27:54.850254+0000","last_fullsized":"2024-02-05T06:27:54.850254+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:45:24.882993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847078+0000","last_change":"2024-02-05T06:27:32.446083+0000","last_active":"2024-02-05T06:27:54.847078+0000","last_peered":"2024-02-05T06:27:54.847078+0000","last_clean":"2024-02-05T06:27:54.847078+0000","last_became_active":"2024-02-05T06:27:32.445450+0000","last_became_peered":"2024-02-05T06:27:32.445450+0000","last_unstale":"2024-02-05T06:27:54.847078+0000","last_undegraded":"2024-02-05T06:27:54.847078+0000","last_fullsized":"2024-02-05T06:27:54.847078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:02:10.288112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444592+0000","last_change":"2024-02-05T06:27:30.422801+0000","last_active":"2024-02-05T06:27:56.444592+0000","last_peered":"2024-02-05T06:27:56.444592+0000","last_clean":"2024-02-05T06:27:56.444592+0000","last_became_active":"2024-02-05T06:27:30.422395+0000","last_became_peered":"2024-02-05T06:27:30.422395+0000","last_unstale":"2024-02-05T06:27:56.444592+0000","last_undegraded":"2024-02-05T06:27:56.444592+0000","last_fullsized":"2024-02-05T06:27:56.444592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:53:10.663620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849402+0000","last_change":"2024-02-05T06:27:26.398022+0000","last_active":"2024-02-05T06:27:54.849402+0000","last_peered":"2024-02-05T06:27:54.849402+0000","last_clean":"2024-02-05T06:27:54.849402+0000","last_became_active":"2024-02-05T06:27:26.397692+0000","last_became_peered":"2024-02-05T06:27:26.397692+0000","last_unstale":"2024-02-05T06:27:54.849402+0000","last_undegraded":"2024-02-05T06:27:54.849402+0000","last_fullsized":"2024-02-05T06:27:54.849402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:19:26.852320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"31'19","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849570+0000","last_change":"2024-02-05T06:27:28.409122+0000","last_active":"2024-02-05T06:27:54.849570+0000","last_peered":"2024-02-05T06:27:54.849570+0000","last_clean":"2024-02-05T06:27:54.849570+0000","last_became_active":"2024-02-05T06:27:28.408759+0000","last_became_peered":"2024-02-05T06:27:28.408759+0000","last_unstale":"2024-02-05T06:27:54.849570+0000","last_undegraded":"2024-02-05T06:27:54.849570+0000","last_fullsized":"2024-02-05T06:27:54.849570+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:57:15.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848408+0000","last_change":"2024-02-05T06:27:32.445655+0000","last_active":"2024-02-05T06:27:54.848408+0000","last_peered":"2024-02-05T06:27:54.848408+0000","last_clean":"2024-02-05T06:27:54.848408+0000","last_became_active":"2024-02-05T06:27:32.445349+0000","last_became_peered":"2024-02-05T06:27:32.445349+0000","last_unstale":"2024-02-05T06:27:54.848408+0000","last_undegraded":"2024-02-05T06:27:54.848408+0000","last_fullsized":"2024-02-05T06:27:54.848408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:47:19.903620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850007+0000","last_change":"2024-02-05T06:27:26.399604+0000","last_active":"2024-02-05T06:27:54.850007+0000","last_peered":"2024-02-05T06:27:54.850007+0000","last_clean":"2024-02-05T06:27:54.850007+0000","last_became_active":"2024-02-05T06:27:26.399145+0000","last_became_peered":"2024-02-05T06:27:26.399145+0000","last_unstale":"2024-02-05T06:27:54.850007+0000","last_undegraded":"2024-02-05T06:27:54.850007+0000","last_fullsized":"2024-02-05T06:27:54.850007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:49:34.114145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"31'17","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850654+0000","last_change":"2024-02-05T06:27:28.410240+0000","last_active":"2024-02-05T06:27:54.850654+0000","last_peered":"2024-02-05T06:27:54.850654+0000","last_clean":"2024-02-05T06:27:54.850654+0000","last_became_active":"2024-02-05T06:27:28.409858+0000","last_became_peered":"2024-02-05T06:27:28.409858+0000","last_unstale":"2024-02-05T06:27:54.850654+0000","last_undegraded":"2024-02-05T06:27:54.850654+0000","last_fullsized":"2024-02-05T06:27:54.850654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:50:31.223880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849290+0000","last_change":"2024-02-05T06:27:30.425854+0000","last_active":"2024-02-05T06:27:54.849290+0000","last_peered":"2024-02-05T06:27:54.849290+0000","last_clean":"2024-02-05T06:27:54.849290+0000","last_became_active":"2024-02-05T06:27:30.424265+0000","last_became_peered":"2024-02-05T06:27:30.424265+0000","last_unstale":"2024-02-05T06:27:54.849290+0000","last_undegraded":"2024-02-05T06:27:54.849290+0000","last_fullsized":"2024-02-05T06:27:54.849290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:59:52.233110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847000+0000","last_change":"2024-02-05T06:27:32.445271+0000","last_active":"2024-02-05T06:27:54.847000+0000","last_peered":"2024-02-05T06:27:54.847000+0000","last_clean":"2024-02-05T06:27:54.847000+0000","last_became_active":"2024-02-05T06:27:32.444920+0000","last_became_peered":"2024-02-05T06:27:32.444920+0000","last_unstale":"2024-02-05T06:27:54.847000+0000","last_undegraded":"2024-02-05T06:27:54.847000+0000","last_fullsized":"2024-02-05T06:27:54.847000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:26:18.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849896+0000","last_change":"2024-02-05T06:27:26.399446+0000","last_active":"2024-02-05T06:27:54.849896+0000","last_peered":"2024-02-05T06:27:54.849896+0000","last_clean":"2024-02-05T06:27:54.849896+0000","last_became_active":"2024-02-05T06:27:26.399162+0000","last_became_peered":"2024-02-05T06:27:26.399162+0000","last_unstale":"2024-02-05T06:27:54.849896+0000","last_undegraded":"2024-02-05T06:27:54.849896+0000","last_fullsized":"2024-02-05T06:27:54.849896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:36:23.180118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850624+0000","last_change":"2024-02-05T06:27:28.410597+0000","last_active":"2024-02-05T06:27:54.850624+0000","last_peered":"2024-02-05T06:27:54.850624+0000","last_clean":"2024-02-05T06:27:54.850624+0000","last_became_active":"2024-02-05T06:27:28.410011+0000","last_became_peered":"2024-02-05T06:27:28.410011+0000","last_unstale":"2024-02-05T06:27:54.850624+0000","last_undegraded":"2024-02-05T06:27:54.850624+0000","last_fullsized":"2024-02-05T06:27:54.850624+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:11:02.504562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847714+0000","last_change":"2024-02-05T06:27:30.430793+0000","last_active":"2024-02-05T06:27:54.847714+0000","last_peered":"2024-02-05T06:27:54.847714+0000","last_clean":"2024-02-05T06:27:54.847714+0000","last_became_active":"2024-02-05T06:27:30.430134+0000","last_became_peered":"2024-02-05T06:27:30.430134+0000","last_unstale":"2024-02-05T06:27:54.847714+0000","last_undegraded":"2024-02-05T06:27:54.847714+0000","last_fullsized":"2024-02-05T06:27:54.847714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:08:57.287272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848787+0000","last_change":"2024-02-05T06:27:32.442752+0000","last_active":"2024-02-05T06:27:54.848787+0000","last_peered":"2024-02-05T06:27:54.848787+0000","last_clean":"2024-02-05T06:27:54.848787+0000","last_became_active":"2024-02-05T06:27:32.441194+0000","last_became_peered":"2024-02-05T06:27:32.441194+0000","last_unstale":"2024-02-05T06:27:54.848787+0000","last_undegraded":"2024-02-05T06:27:54.848787+0000","last_fullsized":"2024-02-05T06:27:54.848787+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:01:37.958119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850282+0000","last_change":"2024-02-05T06:27:26.397248+0000","last_active":"2024-02-05T06:27:54.850282+0000","last_peered":"2024-02-05T06:27:54.850282+0000","last_clean":"2024-02-05T06:27:54.850282+0000","last_became_active":"2024-02-05T06:27:26.395102+0000","last_became_peered":"2024-02-05T06:27:26.395102+0000","last_unstale":"2024-02-05T06:27:54.850282+0000","last_undegraded":"2024-02-05T06:27:54.850282+0000","last_fullsized":"2024-02-05T06:27:54.850282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:53:27.344825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851732+0000","last_change":"2024-02-05T06:27:28.411647+0000","last_active":"2024-02-05T06:27:54.851732+0000","last_peered":"2024-02-05T06:27:54.851732+0000","last_clean":"2024-02-05T06:27:54.851732+0000","last_became_active":"2024-02-05T06:27:28.411113+0000","last_became_peered":"2024-02-05T06:27:28.411113+0000","last_unstale":"2024-02-05T06:27:54.851732+0000","last_undegraded":"2024-02-05T06:27:54.851732+0000","last_fullsized":"2024-02-05T06:27:54.851732+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:05:05.481584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849288+0000","last_change":"2024-02-05T06:27:30.430446+0000","last_active":"2024-02-05T06:27:54.849288+0000","last_peered":"2024-02-05T06:27:54.849288+0000","last_clean":"2024-02-05T06:27:54.849288+0000","last_became_active":"2024-02-05T06:27:30.430109+0000","last_became_peered":"2024-02-05T06:27:30.430109+0000","last_unstale":"2024-02-05T06:27:54.849288+0000","last_undegraded":"2024-02-05T06:27:54.849288+0000","last_fullsized":"2024-02-05T06:27:54.849288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:13:22.266437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708840,"statfs":{"total":95990841344,"available":95957852160,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 06:27:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 06:27:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T06:28:07.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T06:28:08.248 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:08.311 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:08 smithi142 ceph-mon[30053]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T06:28:08.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:08 smithi142 ceph-mon[30053]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:08.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:08 smithi122 ceph-mon[25512]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T06:28:08.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:08 smithi122 ceph-mon[25512]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:08.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:08 smithi196 ceph-mon[30516]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T06:28:08.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:08 smithi196 ceph-mon[30516]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:09.508 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:09 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ::ffff:172.21.15.142 - - [05/Feb/2024:06:28:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T06:28:10.223 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:10 smithi122 ceph-mon[25512]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:10.223 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:10 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:10.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:10 smithi142 ceph-mon[30053]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:10.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:10 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:10.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:10 smithi196 ceph-mon[30516]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:10.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:10 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:10.796 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:28:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-02-05T06:28:11.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:11 smithi142 ceph-mon[30053]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:11.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:11 smithi122 ceph-mon[25512]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:11.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:11 smithi196 ceph-mon[30516]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T06:28:12.229 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-02-05T06:28:08.859202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96640,"kb_used_data":3108,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126528,"statfs":{"total":287972524032,"available":287873564672,"internally_reserved":0,"allocated":3182592,"data_stored":2109330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002382"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849327+0000","last_change":"2024-02-05T06:27:32.440996+0000","last_active":"2024-02-05T06:27:54.849327+0000","last_peered":"2024-02-05T06:27:54.849327+0000","last_clean":"2024-02-05T06:27:54.849327+0000","last_became_active":"2024-02-05T06:27:32.440036+0000","last_became_peered":"2024-02-05T06:27:32.440036+0000","last_unstale":"2024-02-05T06:27:54.849327+0000","last_undegraded":"2024-02-05T06:27:54.849327+0000","last_fullsized":"2024-02-05T06:27:54.849327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:44:43.677631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848082+0000","last_change":"2024-02-05T06:27:30.428443+0000","last_active":"2024-02-05T06:27:54.848082+0000","last_peered":"2024-02-05T06:27:54.848082+0000","last_clean":"2024-02-05T06:27:54.848082+0000","last_became_active":"2024-02-05T06:27:30.428077+0000","last_became_peered":"2024-02-05T06:27:30.428077+0000","last_unstale":"2024-02-05T06:27:54.848082+0000","last_undegraded":"2024-02-05T06:27:54.848082+0000","last_fullsized":"2024-02-05T06:27:54.848082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:07:30.992690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850398+0000","last_change":"2024-02-05T06:27:26.396286+0000","last_active":"2024-02-05T06:27:54.850398+0000","last_peered":"2024-02-05T06:27:54.850398+0000","last_clean":"2024-02-05T06:27:54.850398+0000","last_became_active":"2024-02-05T06:27:26.394815+0000","last_became_peered":"2024-02-05T06:27:26.394815+0000","last_unstale":"2024-02-05T06:27:54.850398+0000","last_undegraded":"2024-02-05T06:27:54.850398+0000","last_fullsized":"2024-02-05T06:27:54.850398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:01:07.273460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848756+0000","last_change":"2024-02-05T06:27:28.407817+0000","last_active":"2024-02-05T06:27:54.848756+0000","last_peered":"2024-02-05T06:27:54.848756+0000","last_clean":"2024-02-05T06:27:54.848756+0000","last_became_active":"2024-02-05T06:27:28.407415+0000","last_became_peered":"2024-02-05T06:27:28.407415+0000","last_unstale":"2024-02-05T06:27:54.848756+0000","last_undegraded":"2024-02-05T06:27:54.848756+0000","last_fullsized":"2024-02-05T06:27:54.848756+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:33:08.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847325+0000","last_change":"2024-02-05T06:27:32.446292+0000","last_active":"2024-02-05T06:27:54.847325+0000","last_peered":"2024-02-05T06:27:54.847325+0000","last_clean":"2024-02-05T06:27:54.847325+0000","last_became_active":"2024-02-05T06:27:32.445995+0000","last_became_peered":"2024-02-05T06:27:32.445995+0000","last_unstale":"2024-02-05T06:27:54.847325+0000","last_undegraded":"2024-02-05T06:27:54.847325+0000","last_fullsized":"2024-02-05T06:27:54.847325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:57:33.643235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849020+0000","last_change":"2024-02-05T06:27:30.426001+0000","last_active":"2024-02-05T06:27:54.849020+0000","last_peered":"2024-02-05T06:27:54.849020+0000","last_clean":"2024-02-05T06:27:54.849020+0000","last_became_active":"2024-02-05T06:27:30.424139+0000","last_became_peered":"2024-02-05T06:27:30.424139+0000","last_unstale":"2024-02-05T06:27:54.849020+0000","last_undegraded":"2024-02-05T06:27:54.849020+0000","last_fullsized":"2024-02-05T06:27:54.849020+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:35:29.851130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849872+0000","last_change":"2024-02-05T06:27:26.394370+0000","last_active":"2024-02-05T06:27:54.849872+0000","last_peered":"2024-02-05T06:27:54.849872+0000","last_clean":"2024-02-05T06:27:54.849872+0000","last_became_active":"2024-02-05T06:27:26.393417+0000","last_became_peered":"2024-02-05T06:27:26.393417+0000","last_unstale":"2024-02-05T06:27:54.849872+0000","last_undegraded":"2024-02-05T06:27:54.849872+0000","last_fullsized":"2024-02-05T06:27:54.849872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:56:00.610220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850205+0000","last_change":"2024-02-05T06:27:28.409096+0000","last_active":"2024-02-05T06:27:54.850205+0000","last_peered":"2024-02-05T06:27:54.850205+0000","last_clean":"2024-02-05T06:27:54.850205+0000","last_became_active":"2024-02-05T06:27:28.407786+0000","last_became_peered":"2024-02-05T06:27:28.407786+0000","last_unstale":"2024-02-05T06:27:54.850205+0000","last_undegraded":"2024-02-05T06:27:54.850205+0000","last_fullsized":"2024-02-05T06:27:54.850205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:42:59.049933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848201+0000","last_change":"2024-02-05T06:27:32.445006+0000","last_active":"2024-02-05T06:27:54.848201+0000","last_peered":"2024-02-05T06:27:54.848201+0000","last_clean":"2024-02-05T06:27:54.848201+0000","last_became_active":"2024-02-05T06:27:32.444708+0000","last_became_peered":"2024-02-05T06:27:32.444708+0000","last_unstale":"2024-02-05T06:27:54.848201+0000","last_undegraded":"2024-02-05T06:27:54.848201+0000","last_fullsized":"2024-02-05T06:27:54.848201+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:22:52.917390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849870+0000","last_change":"2024-02-05T06:27:30.429706+0000","last_active":"2024-02-05T06:27:54.849870+0000","last_peered":"2024-02-05T06:27:54.849870+0000","last_clean":"2024-02-05T06:27:54.849870+0000","last_became_active":"2024-02-05T06:27:30.428198+0000","last_became_peered":"2024-02-05T06:27:30.428198+0000","last_unstale":"2024-02-05T06:27:54.849870+0000","last_undegraded":"2024-02-05T06:27:54.849870+0000","last_fullsized":"2024-02-05T06:27:54.849870+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:39:10.169139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851469+0000","last_change":"2024-02-05T06:27:26.397872+0000","last_active":"2024-02-05T06:27:54.851469+0000","last_peered":"2024-02-05T06:27:54.851469+0000","last_clean":"2024-02-05T06:27:54.851469+0000","last_became_active":"2024-02-05T06:27:26.397562+0000","last_became_peered":"2024-02-05T06:27:26.397562+0000","last_unstale":"2024-02-05T06:27:54.851469+0000","last_undegraded":"2024-02-05T06:27:54.851469+0000","last_fullsized":"2024-02-05T06:27:54.851469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:03:03.690389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851811+0000","last_change":"2024-02-05T06:27:28.410333+0000","last_active":"2024-02-05T06:27:54.851811+0000","last_peered":"2024-02-05T06:27:54.851811+0000","last_clean":"2024-02-05T06:27:54.851811+0000","last_became_active":"2024-02-05T06:27:28.409958+0000","last_became_peered":"2024-02-05T06:27:28.409958+0000","last_unstale":"2024-02-05T06:27:54.851811+0000","last_undegraded":"2024-02-05T06:27:54.851811+0000","last_fullsized":"2024-02-05T06:27:54.851811+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:53:32.103202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849745+0000","last_change":"2024-02-05T06:27:32.445983+0000","last_active":"2024-02-05T06:27:54.849745+0000","last_peered":"2024-02-05T06:27:54.849745+0000","last_clean":"2024-02-05T06:27:54.849745+0000","last_became_active":"2024-02-05T06:27:32.445609+0000","last_became_peered":"2024-02-05T06:27:32.445609+0000","last_unstale":"2024-02-05T06:27:54.849745+0000","last_undegraded":"2024-02-05T06:27:54.849745+0000","last_fullsized":"2024-02-05T06:27:54.849745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:17:30.014715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849140+0000","last_change":"2024-02-05T06:27:30.421761+0000","last_active":"2024-02-05T06:27:54.849140+0000","last_peered":"2024-02-05T06:27:54.849140+0000","last_clean":"2024-02-05T06:27:54.849140+0000","last_became_active":"2024-02-05T06:27:30.421177+0000","last_became_peered":"2024-02-05T06:27:30.421177+0000","last_unstale":"2024-02-05T06:27:54.849140+0000","last_undegraded":"2024-02-05T06:27:54.849140+0000","last_fullsized":"2024-02-05T06:27:54.849140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:37:08.277435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849599+0000","last_change":"2024-02-05T06:27:26.398670+0000","last_active":"2024-02-05T06:27:54.849599+0000","last_peered":"2024-02-05T06:27:54.849599+0000","last_clean":"2024-02-05T06:27:54.849599+0000","last_became_active":"2024-02-05T06:27:26.398148+0000","last_became_peered":"2024-02-05T06:27:26.398148+0000","last_unstale":"2024-02-05T06:27:54.849599+0000","last_undegraded":"2024-02-05T06:27:54.849599+0000","last_fullsized":"2024-02-05T06:27:54.849599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:17:55.417724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851381+0000","last_change":"2024-02-05T06:27:28.413706+0000","last_active":"2024-02-05T06:27:54.851381+0000","last_peered":"2024-02-05T06:27:54.851381+0000","last_clean":"2024-02-05T06:27:54.851381+0000","last_became_active":"2024-02-05T06:27:28.413431+0000","last_became_peered":"2024-02-05T06:27:28.413431+0000","last_unstale":"2024-02-05T06:27:54.851381+0000","last_undegraded":"2024-02-05T06:27:54.851381+0000","last_fullsized":"2024-02-05T06:27:54.851381+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:34:22.164454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850149+0000","last_change":"2024-02-05T06:27:32.442862+0000","last_active":"2024-02-05T06:27:54.850149+0000","last_peered":"2024-02-05T06:27:54.850149+0000","last_clean":"2024-02-05T06:27:54.850149+0000","last_became_active":"2024-02-05T06:27:32.441713+0000","last_became_peered":"2024-02-05T06:27:32.441713+0000","last_unstale":"2024-02-05T06:27:54.850149+0000","last_undegraded":"2024-02-05T06:27:54.850149+0000","last_fullsized":"2024-02-05T06:27:54.850149+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:22:41.483099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848933+0000","last_change":"2024-02-05T06:27:30.428018+0000","last_active":"2024-02-05T06:27:54.848933+0000","last_peered":"2024-02-05T06:27:54.848933+0000","last_clean":"2024-02-05T06:27:54.848933+0000","last_became_active":"2024-02-05T06:27:30.427656+0000","last_became_peered":"2024-02-05T06:27:30.427656+0000","last_unstale":"2024-02-05T06:27:54.848933+0000","last_undegraded":"2024-02-05T06:27:54.848933+0000","last_fullsized":"2024-02-05T06:27:54.848933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:05:58.607809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849599+0000","last_change":"2024-02-05T06:27:26.398312+0000","last_active":"2024-02-05T06:27:54.849599+0000","last_peered":"2024-02-05T06:27:54.849599+0000","last_clean":"2024-02-05T06:27:54.849599+0000","last_became_active":"2024-02-05T06:27:26.397665+0000","last_became_peered":"2024-02-05T06:27:26.397665+0000","last_unstale":"2024-02-05T06:27:54.849599+0000","last_undegraded":"2024-02-05T06:27:54.849599+0000","last_fullsized":"2024-02-05T06:27:54.849599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:07:06.141059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849164+0000","last_change":"2024-02-05T06:27:28.412998+0000","last_active":"2024-02-05T06:27:54.849164+0000","last_peered":"2024-02-05T06:27:54.849164+0000","last_clean":"2024-02-05T06:27:54.849164+0000","last_became_active":"2024-02-05T06:27:28.412681+0000","last_became_peered":"2024-02-05T06:27:28.412681+0000","last_unstale":"2024-02-05T06:27:54.849164+0000","last_undegraded":"2024-02-05T06:27:54.849164+0000","last_fullsized":"2024-02-05T06:27:54.849164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:07:13.536299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848496+0000","last_change":"2024-02-05T06:27:32.447131+0000","last_active":"2024-02-05T06:27:54.848496+0000","last_peered":"2024-02-05T06:27:54.848496+0000","last_clean":"2024-02-05T06:27:54.848496+0000","last_became_active":"2024-02-05T06:27:32.446666+0000","last_became_peered":"2024-02-05T06:27:32.446666+0000","last_unstale":"2024-02-05T06:27:54.848496+0000","last_undegraded":"2024-02-05T06:27:54.848496+0000","last_fullsized":"2024-02-05T06:27:54.848496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:35:00.776937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849686+0000","last_change":"2024-02-05T06:27:30.427419+0000","last_active":"2024-02-05T06:27:54.849686+0000","last_peered":"2024-02-05T06:27:54.849686+0000","last_clean":"2024-02-05T06:27:54.849686+0000","last_became_active":"2024-02-05T06:27:30.426648+0000","last_became_peered":"2024-02-05T06:27:30.426648+0000","last_unstale":"2024-02-05T06:27:54.849686+0000","last_undegraded":"2024-02-05T06:27:54.849686+0000","last_fullsized":"2024-02-05T06:27:54.849686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:22:24.775157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849512+0000","last_change":"2024-02-05T06:27:26.397528+0000","last_active":"2024-02-05T06:27:54.849512+0000","last_peered":"2024-02-05T06:27:54.849512+0000","last_clean":"2024-02-05T06:27:54.849512+0000","last_became_active":"2024-02-05T06:27:26.396050+0000","last_became_peered":"2024-02-05T06:27:26.396050+0000","last_unstale":"2024-02-05T06:27:54.849512+0000","last_undegraded":"2024-02-05T06:27:54.849512+0000","last_fullsized":"2024-02-05T06:27:54.849512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:07:32.642538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"31'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848998+0000","last_change":"2024-02-05T06:27:28.412601+0000","last_active":"2024-02-05T06:27:54.848998+0000","last_peered":"2024-02-05T06:27:54.848998+0000","last_clean":"2024-02-05T06:27:54.848998+0000","last_became_active":"2024-02-05T06:27:28.411748+0000","last_became_peered":"2024-02-05T06:27:28.411748+0000","last_unstale":"2024-02-05T06:27:54.848998+0000","last_undegraded":"2024-02-05T06:27:54.848998+0000","last_fullsized":"2024-02-05T06:27:54.848998+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:49.872034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848141+0000","last_change":"2024-02-05T06:27:32.448682+0000","last_active":"2024-02-05T06:27:54.848141+0000","last_peered":"2024-02-05T06:27:54.848141+0000","last_clean":"2024-02-05T06:27:54.848141+0000","last_became_active":"2024-02-05T06:27:32.447956+0000","last_became_peered":"2024-02-05T06:27:32.447956+0000","last_unstale":"2024-02-05T06:27:54.848141+0000","last_undegraded":"2024-02-05T06:27:54.848141+0000","last_fullsized":"2024-02-05T06:27:54.848141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:42:50.345955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'8","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444475+0000","last_change":"2024-02-05T06:27:30.428005+0000","last_active":"2024-02-05T06:27:56.444475+0000","last_peered":"2024-02-05T06:27:56.444475+0000","last_clean":"2024-02-05T06:27:56.444475+0000","last_became_active":"2024-02-05T06:27:30.427144+0000","last_became_peered":"2024-02-05T06:27:30.427144+0000","last_unstale":"2024-02-05T06:27:56.444475+0000","last_undegraded":"2024-02-05T06:27:56.444475+0000","last_fullsized":"2024-02-05T06:27:56.444475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:49:49.138800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847845+0000","last_change":"2024-02-05T06:27:26.396069+0000","last_active":"2024-02-05T06:27:54.847845+0000","last_peered":"2024-02-05T06:27:54.847845+0000","last_clean":"2024-02-05T06:27:54.847845+0000","last_became_active":"2024-02-05T06:27:26.395074+0000","last_became_peered":"2024-02-05T06:27:26.395074+0000","last_unstale":"2024-02-05T06:27:54.847845+0000","last_undegraded":"2024-02-05T06:27:54.847845+0000","last_fullsized":"2024-02-05T06:27:54.847845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:32:32.810357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846843+0000","last_change":"2024-02-05T06:27:28.407073+0000","last_active":"2024-02-05T06:27:54.846843+0000","last_peered":"2024-02-05T06:27:54.846843+0000","last_clean":"2024-02-05T06:27:54.846843+0000","last_became_active":"2024-02-05T06:27:28.406720+0000","last_became_peered":"2024-02-05T06:27:28.406720+0000","last_unstale":"2024-02-05T06:27:54.846843+0000","last_undegraded":"2024-02-05T06:27:54.846843+0000","last_fullsized":"2024-02-05T06:27:54.846843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:49:02.062634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850791+0000","last_change":"2024-02-05T06:27:32.446492+0000","last_active":"2024-02-05T06:27:54.850791+0000","last_peered":"2024-02-05T06:27:54.850791+0000","last_clean":"2024-02-05T06:27:54.850791+0000","last_became_active":"2024-02-05T06:27:32.446077+0000","last_became_peered":"2024-02-05T06:27:32.446077+0000","last_unstale":"2024-02-05T06:27:54.850791+0000","last_undegraded":"2024-02-05T06:27:54.850791+0000","last_fullsized":"2024-02-05T06:27:54.850791+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:17:36.979180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848398+0000","last_change":"2024-02-05T06:27:30.430871+0000","last_active":"2024-02-05T06:27:54.848398+0000","last_peered":"2024-02-05T06:27:54.848398+0000","last_clean":"2024-02-05T06:27:54.848398+0000","last_became_active":"2024-02-05T06:27:30.429608+0000","last_became_peered":"2024-02-05T06:27:30.429608+0000","last_unstale":"2024-02-05T06:27:54.848398+0000","last_undegraded":"2024-02-05T06:27:54.848398+0000","last_fullsized":"2024-02-05T06:27:54.848398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:42:58.678081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848340+0000","last_change":"2024-02-05T06:27:26.398260+0000","last_active":"2024-02-05T06:27:54.848340+0000","last_peered":"2024-02-05T06:27:54.848340+0000","last_clean":"2024-02-05T06:27:54.848340+0000","last_became_active":"2024-02-05T06:27:26.397505+0000","last_became_peered":"2024-02-05T06:27:26.397505+0000","last_unstale":"2024-02-05T06:27:54.848340+0000","last_undegraded":"2024-02-05T06:27:54.848340+0000","last_fullsized":"2024-02-05T06:27:54.848340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:48:56.733841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846923+0000","last_change":"2024-02-05T06:27:28.406073+0000","last_active":"2024-02-05T06:27:54.846923+0000","last_peered":"2024-02-05T06:27:54.846923+0000","last_clean":"2024-02-05T06:27:54.846923+0000","last_became_active":"2024-02-05T06:27:28.405310+0000","last_became_peered":"2024-02-05T06:27:28.405310+0000","last_unstale":"2024-02-05T06:27:54.846923+0000","last_undegraded":"2024-02-05T06:27:54.846923+0000","last_fullsized":"2024-02-05T06:27:54.846923+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:21:47.393186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849408+0000","last_change":"2024-02-05T06:27:32.443085+0000","last_active":"2024-02-05T06:27:54.849408+0000","last_peered":"2024-02-05T06:27:54.849408+0000","last_clean":"2024-02-05T06:27:54.849408+0000","last_became_active":"2024-02-05T06:27:32.442638+0000","last_became_peered":"2024-02-05T06:27:32.442638+0000","last_unstale":"2024-02-05T06:27:54.849408+0000","last_undegraded":"2024-02-05T06:27:54.849408+0000","last_fullsized":"2024-02-05T06:27:54.849408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:05:56.215799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.846818+0000","last_change":"2024-02-05T06:27:30.428791+0000","last_active":"2024-02-05T06:27:54.846818+0000","last_peered":"2024-02-05T06:27:54.846818+0000","last_clean":"2024-02-05T06:27:54.846818+0000","last_became_active":"2024-02-05T06:27:30.428397+0000","last_became_peered":"2024-02-05T06:27:30.428397+0000","last_unstale":"2024-02-05T06:27:54.846818+0000","last_undegraded":"2024-02-05T06:27:54.846818+0000","last_fullsized":"2024-02-05T06:27:54.846818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:41:57.810713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851952+0000","last_change":"2024-02-05T06:27:26.397397+0000","last_active":"2024-02-05T06:27:54.851952+0000","last_peered":"2024-02-05T06:27:54.851952+0000","last_clean":"2024-02-05T06:27:54.851952+0000","last_became_active":"2024-02-05T06:27:26.397022+0000","last_became_peered":"2024-02-05T06:27:26.397022+0000","last_unstale":"2024-02-05T06:27:54.851952+0000","last_undegraded":"2024-02-05T06:27:54.851952+0000","last_fullsized":"2024-02-05T06:27:54.851952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:46:38.131979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851836+0000","last_change":"2024-02-05T06:27:28.409173+0000","last_active":"2024-02-05T06:27:54.851836+0000","last_peered":"2024-02-05T06:27:54.851836+0000","last_clean":"2024-02-05T06:27:54.851836+0000","last_became_active":"2024-02-05T06:27:28.408804+0000","last_became_peered":"2024-02-05T06:27:28.408804+0000","last_unstale":"2024-02-05T06:27:54.851836+0000","last_undegraded":"2024-02-05T06:27:54.851836+0000","last_fullsized":"2024-02-05T06:27:54.851836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:28:37.097941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849159+0000","last_change":"2024-02-05T06:27:32.442743+0000","last_active":"2024-02-05T06:27:54.849159+0000","last_peered":"2024-02-05T06:27:54.849159+0000","last_clean":"2024-02-05T06:27:54.849159+0000","last_became_active":"2024-02-05T06:27:32.441600+0000","last_became_peered":"2024-02-05T06:27:32.441600+0000","last_unstale":"2024-02-05T06:27:54.849159+0000","last_undegraded":"2024-02-05T06:27:54.849159+0000","last_fullsized":"2024-02-05T06:27:54.849159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:22:12.036136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849958+0000","last_change":"2024-02-05T06:27:30.423661+0000","last_active":"2024-02-05T06:27:54.849958+0000","last_peered":"2024-02-05T06:27:54.849958+0000","last_clean":"2024-02-05T06:27:54.849958+0000","last_became_active":"2024-02-05T06:27:30.423301+0000","last_became_peered":"2024-02-05T06:27:30.423301+0000","last_unstale":"2024-02-05T06:27:54.849958+0000","last_undegraded":"2024-02-05T06:27:54.849958+0000","last_fullsized":"2024-02-05T06:27:54.849958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:43:11.421123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851103+0000","last_change":"2024-02-05T06:27:26.399601+0000","last_active":"2024-02-05T06:27:54.851103+0000","last_peered":"2024-02-05T06:27:54.851103+0000","last_clean":"2024-02-05T06:27:54.851103+0000","last_became_active":"2024-02-05T06:27:26.399269+0000","last_became_peered":"2024-02-05T06:27:26.399269+0000","last_unstale":"2024-02-05T06:27:54.851103+0000","last_undegraded":"2024-02-05T06:27:54.851103+0000","last_fullsized":"2024-02-05T06:27:54.851103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:33:22.859827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"31'6","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850540+0000","last_change":"2024-02-05T06:27:28.407104+0000","last_active":"2024-02-05T06:27:54.850540+0000","last_peered":"2024-02-05T06:27:54.850540+0000","last_clean":"2024-02-05T06:27:54.850540+0000","last_became_active":"2024-02-05T06:27:28.406801+0000","last_became_peered":"2024-02-05T06:27:28.406801+0000","last_unstale":"2024-02-05T06:27:54.850540+0000","last_undegraded":"2024-02-05T06:27:54.850540+0000","last_fullsized":"2024-02-05T06:27:54.850540+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:36:38.771294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848350+0000","last_change":"2024-02-05T06:27:32.444736+0000","last_active":"2024-02-05T06:27:54.848350+0000","last_peered":"2024-02-05T06:27:54.848350+0000","last_clean":"2024-02-05T06:27:54.848350+0000","last_became_active":"2024-02-05T06:27:32.444404+0000","last_became_peered":"2024-02-05T06:27:32.444404+0000","last_unstale":"2024-02-05T06:27:54.848350+0000","last_undegraded":"2024-02-05T06:27:54.848350+0000","last_fullsized":"2024-02-05T06:27:54.848350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:36:14.991005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849976+0000","last_change":"2024-02-05T06:27:30.429431+0000","last_active":"2024-02-05T06:27:54.849976+0000","last_peered":"2024-02-05T06:27:54.849976+0000","last_clean":"2024-02-05T06:27:54.849976+0000","last_became_active":"2024-02-05T06:27:30.427892+0000","last_became_peered":"2024-02-05T06:27:30.427892+0000","last_unstale":"2024-02-05T06:27:54.849976+0000","last_undegraded":"2024-02-05T06:27:54.849976+0000","last_fullsized":"2024-02-05T06:27:54.849976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:39:24.547014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851323+0000","last_change":"2024-02-05T06:27:26.398124+0000","last_active":"2024-02-05T06:27:54.851323+0000","last_peered":"2024-02-05T06:27:54.851323+0000","last_clean":"2024-02-05T06:27:54.851323+0000","last_became_active":"2024-02-05T06:27:26.397672+0000","last_became_peered":"2024-02-05T06:27:26.397672+0000","last_unstale":"2024-02-05T06:27:54.851323+0000","last_undegraded":"2024-02-05T06:27:54.851323+0000","last_fullsized":"2024-02-05T06:27:54.851323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:41:32.865679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851218+0000","last_change":"2024-02-05T06:27:28.410794+0000","last_active":"2024-02-05T06:27:54.851218+0000","last_peered":"2024-02-05T06:27:54.851218+0000","last_clean":"2024-02-05T06:27:54.851218+0000","last_became_active":"2024-02-05T06:27:28.410427+0000","last_became_peered":"2024-02-05T06:27:28.410427+0000","last_unstale":"2024-02-05T06:27:54.851218+0000","last_undegraded":"2024-02-05T06:27:54.851218+0000","last_fullsized":"2024-02-05T06:27:54.851218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:47:12.935091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848549+0000","last_change":"2024-02-05T06:27:32.445626+0000","last_active":"2024-02-05T06:27:54.848549+0000","last_peered":"2024-02-05T06:27:54.848549+0000","last_clean":"2024-02-05T06:27:54.848549+0000","last_became_active":"2024-02-05T06:27:32.445289+0000","last_became_peered":"2024-02-05T06:27:32.445289+0000","last_unstale":"2024-02-05T06:27:54.848549+0000","last_undegraded":"2024-02-05T06:27:54.848549+0000","last_fullsized":"2024-02-05T06:27:54.848549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:28:04.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847367+0000","last_change":"2024-02-05T06:27:30.429732+0000","last_active":"2024-02-05T06:27:54.847367+0000","last_peered":"2024-02-05T06:27:54.847367+0000","last_clean":"2024-02-05T06:27:54.847367+0000","last_became_active":"2024-02-05T06:27:30.429211+0000","last_became_peered":"2024-02-05T06:27:30.429211+0000","last_unstale":"2024-02-05T06:27:54.847367+0000","last_undegraded":"2024-02-05T06:27:54.847367+0000","last_fullsized":"2024-02-05T06:27:54.847367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:27:23.179396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850452+0000","last_change":"2024-02-05T06:27:26.399156+0000","last_active":"2024-02-05T06:27:54.850452+0000","last_peered":"2024-02-05T06:27:54.850452+0000","last_clean":"2024-02-05T06:27:54.850452+0000","last_became_active":"2024-02-05T06:27:26.398705+0000","last_became_peered":"2024-02-05T06:27:26.398705+0000","last_unstale":"2024-02-05T06:27:54.850452+0000","last_undegraded":"2024-02-05T06:27:54.850452+0000","last_fullsized":"2024-02-05T06:27:54.850452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:30:36.542735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849984+0000","last_change":"2024-02-05T06:27:28.413480+0000","last_active":"2024-02-05T06:27:54.849984+0000","last_peered":"2024-02-05T06:27:54.849984+0000","last_clean":"2024-02-05T06:27:54.849984+0000","last_became_active":"2024-02-05T06:27:28.413206+0000","last_became_peered":"2024-02-05T06:27:28.413206+0000","last_unstale":"2024-02-05T06:27:54.849984+0000","last_undegraded":"2024-02-05T06:27:54.849984+0000","last_fullsized":"2024-02-05T06:27:54.849984+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:29:51.921616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847255+0000","last_change":"2024-02-05T06:27:32.448341+0000","last_active":"2024-02-05T06:27:54.847255+0000","last_peered":"2024-02-05T06:27:54.847255+0000","last_clean":"2024-02-05T06:27:54.847255+0000","last_became_active":"2024-02-05T06:27:32.448001+0000","last_became_peered":"2024-02-05T06:27:32.448001+0000","last_unstale":"2024-02-05T06:27:54.847255+0000","last_undegraded":"2024-02-05T06:27:54.847255+0000","last_fullsized":"2024-02-05T06:27:54.847255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:33:22.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444472+0000","last_change":"2024-02-05T06:27:30.427052+0000","last_active":"2024-02-05T06:27:56.444472+0000","last_peered":"2024-02-05T06:27:56.444472+0000","last_clean":"2024-02-05T06:27:56.444472+0000","last_became_active":"2024-02-05T06:27:30.426680+0000","last_became_peered":"2024-02-05T06:27:30.426680+0000","last_unstale":"2024-02-05T06:27:56.444472+0000","last_undegraded":"2024-02-05T06:27:56.444472+0000","last_fullsized":"2024-02-05T06:27:56.444472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:22:15.771257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848799+0000","last_change":"2024-02-05T06:27:26.400082+0000","last_active":"2024-02-05T06:27:54.848799+0000","last_peered":"2024-02-05T06:27:54.848799+0000","last_clean":"2024-02-05T06:27:54.848799+0000","last_became_active":"2024-02-05T06:27:26.399630+0000","last_became_peered":"2024-02-05T06:27:26.399630+0000","last_unstale":"2024-02-05T06:27:54.848799+0000","last_undegraded":"2024-02-05T06:27:54.848799+0000","last_fullsized":"2024-02-05T06:27:54.848799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:32:33.269707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847725+0000","last_change":"2024-02-05T06:27:28.407218+0000","last_active":"2024-02-05T06:27:54.847725+0000","last_peered":"2024-02-05T06:27:54.847725+0000","last_clean":"2024-02-05T06:27:54.847725+0000","last_became_active":"2024-02-05T06:27:28.405368+0000","last_became_peered":"2024-02-05T06:27:28.405368+0000","last_unstale":"2024-02-05T06:27:54.847725+0000","last_undegraded":"2024-02-05T06:27:54.847725+0000","last_fullsized":"2024-02-05T06:27:54.847725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:13:45.383844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850113+0000","last_change":"2024-02-05T06:27:32.442835+0000","last_active":"2024-02-05T06:27:54.850113+0000","last_peered":"2024-02-05T06:27:54.850113+0000","last_clean":"2024-02-05T06:27:54.850113+0000","last_became_active":"2024-02-05T06:27:32.441343+0000","last_became_peered":"2024-02-05T06:27:32.441343+0000","last_unstale":"2024-02-05T06:27:54.850113+0000","last_undegraded":"2024-02-05T06:27:54.850113+0000","last_fullsized":"2024-02-05T06:27:54.850113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:50:11.176760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444492+0000","last_change":"2024-02-05T06:27:30.428355+0000","last_active":"2024-02-05T06:27:56.444492+0000","last_peered":"2024-02-05T06:27:56.444492+0000","last_clean":"2024-02-05T06:27:56.444492+0000","last_became_active":"2024-02-05T06:27:30.428000+0000","last_became_peered":"2024-02-05T06:27:30.428000+0000","last_unstale":"2024-02-05T06:27:56.444492+0000","last_undegraded":"2024-02-05T06:27:56.444492+0000","last_fullsized":"2024-02-05T06:27:56.444492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:41:10.850477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848455+0000","last_change":"2024-02-05T06:27:26.401154+0000","last_active":"2024-02-05T06:27:54.848455+0000","last_peered":"2024-02-05T06:27:54.848455+0000","last_clean":"2024-02-05T06:27:54.848455+0000","last_became_active":"2024-02-05T06:27:26.400337+0000","last_became_peered":"2024-02-05T06:27:26.400337+0000","last_unstale":"2024-02-05T06:27:54.848455+0000","last_undegraded":"2024-02-05T06:27:54.848455+0000","last_fullsized":"2024-02-05T06:27:54.848455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:50:16.332499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850342+0000","last_change":"2024-02-05T06:27:28.408308+0000","last_active":"2024-02-05T06:27:54.850342+0000","last_peered":"2024-02-05T06:27:54.850342+0000","last_clean":"2024-02-05T06:27:54.850342+0000","last_became_active":"2024-02-05T06:27:28.407935+0000","last_became_peered":"2024-02-05T06:27:28.407935+0000","last_unstale":"2024-02-05T06:27:54.850342+0000","last_undegraded":"2024-02-05T06:27:54.850342+0000","last_fullsized":"2024-02-05T06:27:54.850342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:45:56.352527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850593+0000","last_change":"2024-02-05T06:27:32.442800+0000","last_active":"2024-02-05T06:27:54.850593+0000","last_peered":"2024-02-05T06:27:54.850593+0000","last_clean":"2024-02-05T06:27:54.850593+0000","last_became_active":"2024-02-05T06:27:32.441702+0000","last_became_peered":"2024-02-05T06:27:32.441702+0000","last_unstale":"2024-02-05T06:27:54.850593+0000","last_undegraded":"2024-02-05T06:27:54.850593+0000","last_fullsized":"2024-02-05T06:27:54.850593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:13:02.611403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850046+0000","last_change":"2024-02-05T06:27:30.430589+0000","last_active":"2024-02-05T06:27:54.850046+0000","last_peered":"2024-02-05T06:27:54.850046+0000","last_clean":"2024-02-05T06:27:54.850046+0000","last_became_active":"2024-02-05T06:27:30.429718+0000","last_became_peered":"2024-02-05T06:27:30.429718+0000","last_unstale":"2024-02-05T06:27:54.850046+0000","last_undegraded":"2024-02-05T06:27:54.850046+0000","last_fullsized":"2024-02-05T06:27:54.850046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:30:31.957515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849054+0000","last_change":"2024-02-05T06:27:26.395707+0000","last_active":"2024-02-05T06:27:54.849054+0000","last_peered":"2024-02-05T06:27:54.849054+0000","last_clean":"2024-02-05T06:27:54.849054+0000","last_became_active":"2024-02-05T06:27:26.395402+0000","last_became_peered":"2024-02-05T06:27:26.395402+0000","last_unstale":"2024-02-05T06:27:54.849054+0000","last_undegraded":"2024-02-05T06:27:54.849054+0000","last_fullsized":"2024-02-05T06:27:54.849054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:47:08.789979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"31'4","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848659+0000","last_change":"2024-02-05T06:27:28.409566+0000","last_active":"2024-02-05T06:27:54.848659+0000","last_peered":"2024-02-05T06:27:54.848659+0000","last_clean":"2024-02-05T06:27:54.848659+0000","last_became_active":"2024-02-05T06:27:28.408853+0000","last_became_peered":"2024-02-05T06:27:28.408853+0000","last_unstale":"2024-02-05T06:27:54.848659+0000","last_undegraded":"2024-02-05T06:27:54.848659+0000","last_fullsized":"2024-02-05T06:27:54.848659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:35:37.974941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848738+0000","last_change":"2024-02-05T06:27:32.447223+0000","last_active":"2024-02-05T06:27:54.848738+0000","last_peered":"2024-02-05T06:27:54.848738+0000","last_clean":"2024-02-05T06:27:54.848738+0000","last_became_active":"2024-02-05T06:27:32.446866+0000","last_became_peered":"2024-02-05T06:27:32.446866+0000","last_unstale":"2024-02-05T06:27:54.848738+0000","last_undegraded":"2024-02-05T06:27:54.848738+0000","last_fullsized":"2024-02-05T06:27:54.848738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:17:12.028749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848482+0000","last_change":"2024-02-05T06:27:30.429780+0000","last_active":"2024-02-05T06:27:54.848482+0000","last_peered":"2024-02-05T06:27:54.848482+0000","last_clean":"2024-02-05T06:27:54.848482+0000","last_became_active":"2024-02-05T06:27:30.429275+0000","last_became_peered":"2024-02-05T06:27:30.429275+0000","last_unstale":"2024-02-05T06:27:54.848482+0000","last_undegraded":"2024-02-05T06:27:54.848482+0000","last_fullsized":"2024-02-05T06:27:54.848482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:56:40.686628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848501+0000","last_change":"2024-02-05T06:27:26.401129+0000","last_active":"2024-02-05T06:27:54.848501+0000","last_peered":"2024-02-05T06:27:54.848501+0000","last_clean":"2024-02-05T06:27:54.848501+0000","last_became_active":"2024-02-05T06:27:26.400591+0000","last_became_peered":"2024-02-05T06:27:26.400591+0000","last_unstale":"2024-02-05T06:27:54.848501+0000","last_undegraded":"2024-02-05T06:27:54.848501+0000","last_fullsized":"2024-02-05T06:27:54.848501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:56:01.716537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848869+0000","last_change":"2024-02-05T06:27:28.412546+0000","last_active":"2024-02-05T06:27:54.848869+0000","last_peered":"2024-02-05T06:27:54.848869+0000","last_clean":"2024-02-05T06:27:54.848869+0000","last_became_active":"2024-02-05T06:27:28.412211+0000","last_became_peered":"2024-02-05T06:27:28.412211+0000","last_unstale":"2024-02-05T06:27:54.848869+0000","last_undegraded":"2024-02-05T06:27:54.848869+0000","last_fullsized":"2024-02-05T06:27:54.848869+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:54:51.755993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849828+0000","last_change":"2024-02-05T06:27:32.449484+0000","last_active":"2024-02-05T06:27:54.849828+0000","last_peered":"2024-02-05T06:27:54.849828+0000","last_clean":"2024-02-05T06:27:54.849828+0000","last_became_active":"2024-02-05T06:27:32.449190+0000","last_became_peered":"2024-02-05T06:27:32.449190+0000","last_unstale":"2024-02-05T06:27:54.849828+0000","last_undegraded":"2024-02-05T06:27:54.849828+0000","last_fullsized":"2024-02-05T06:27:54.849828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:19:12.198205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851223+0000","last_change":"2024-02-05T06:27:26.399240+0000","last_active":"2024-02-05T06:27:54.851223+0000","last_peered":"2024-02-05T06:27:54.851223+0000","last_clean":"2024-02-05T06:27:54.851223+0000","last_became_active":"2024-02-05T06:27:26.398837+0000","last_became_peered":"2024-02-05T06:27:26.398837+0000","last_unstale":"2024-02-05T06:27:54.851223+0000","last_undegraded":"2024-02-05T06:27:54.851223+0000","last_fullsized":"2024-02-05T06:27:54.851223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:13:16.694898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849932+0000","last_change":"2024-02-05T06:27:28.413614+0000","last_active":"2024-02-05T06:27:54.849932+0000","last_peered":"2024-02-05T06:27:54.849932+0000","last_clean":"2024-02-05T06:27:54.849932+0000","last_became_active":"2024-02-05T06:27:28.413323+0000","last_became_peered":"2024-02-05T06:27:28.413323+0000","last_unstale":"2024-02-05T06:27:54.849932+0000","last_undegraded":"2024-02-05T06:27:54.849932+0000","last_fullsized":"2024-02-05T06:27:54.849932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:43:59.406472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'8","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444533+0000","last_change":"2024-02-05T06:27:30.427941+0000","last_active":"2024-02-05T06:27:56.444533+0000","last_peered":"2024-02-05T06:27:56.444533+0000","last_clean":"2024-02-05T06:27:56.444533+0000","last_became_active":"2024-02-05T06:27:30.427210+0000","last_became_peered":"2024-02-05T06:27:30.427210+0000","last_unstale":"2024-02-05T06:27:56.444533+0000","last_undegraded":"2024-02-05T06:27:56.444533+0000","last_fullsized":"2024-02-05T06:27:56.444533+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:06:14.803220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847183+0000","last_change":"2024-02-05T06:27:32.449362+0000","last_active":"2024-02-05T06:27:54.847183+0000","last_peered":"2024-02-05T06:27:54.847183+0000","last_clean":"2024-02-05T06:27:54.847183+0000","last_became_active":"2024-02-05T06:27:32.449094+0000","last_became_peered":"2024-02-05T06:27:32.449094+0000","last_unstale":"2024-02-05T06:27:54.847183+0000","last_undegraded":"2024-02-05T06:27:54.847183+0000","last_fullsized":"2024-02-05T06:27:54.847183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:18.263817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848455+0000","last_change":"2024-02-05T06:27:26.393500+0000","last_active":"2024-02-05T06:27:54.848455+0000","last_peered":"2024-02-05T06:27:54.848455+0000","last_clean":"2024-02-05T06:27:54.848455+0000","last_became_active":"2024-02-05T06:27:26.392118+0000","last_became_peered":"2024-02-05T06:27:26.392118+0000","last_unstale":"2024-02-05T06:27:54.848455+0000","last_undegraded":"2024-02-05T06:27:54.848455+0000","last_fullsized":"2024-02-05T06:27:54.848455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:04:52.273437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"31'16","reported_seq":53,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444592+0000","last_change":"2024-02-05T06:27:28.409328+0000","last_active":"2024-02-05T06:27:56.444592+0000","last_peered":"2024-02-05T06:27:56.444592+0000","last_clean":"2024-02-05T06:27:56.444592+0000","last_became_active":"2024-02-05T06:27:28.409003+0000","last_became_peered":"2024-02-05T06:27:28.409003+0000","last_unstale":"2024-02-05T06:27:56.444592+0000","last_undegraded":"2024-02-05T06:27:56.444592+0000","last_fullsized":"2024-02-05T06:27:56.444592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:42:32.044272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848371+0000","last_change":"2024-02-05T06:27:30.423065+0000","last_active":"2024-02-05T06:27:54.848371+0000","last_peered":"2024-02-05T06:27:54.848371+0000","last_clean":"2024-02-05T06:27:54.848371+0000","last_became_active":"2024-02-05T06:27:30.422780+0000","last_became_peered":"2024-02-05T06:27:30.422780+0000","last_unstale":"2024-02-05T06:27:54.848371+0000","last_undegraded":"2024-02-05T06:27:54.848371+0000","last_fullsized":"2024-02-05T06:27:54.848371+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:45:46.674581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847701+0000","last_change":"2024-02-05T06:27:32.447396+0000","last_active":"2024-02-05T06:27:54.847701+0000","last_peered":"2024-02-05T06:27:54.847701+0000","last_clean":"2024-02-05T06:27:54.847701+0000","last_became_active":"2024-02-05T06:27:32.446490+0000","last_became_peered":"2024-02-05T06:27:32.446490+0000","last_unstale":"2024-02-05T06:27:54.847701+0000","last_undegraded":"2024-02-05T06:27:54.847701+0000","last_fullsized":"2024-02-05T06:27:54.847701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:15:28.025147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849410+0000","last_change":"2024-02-05T06:27:26.397713+0000","last_active":"2024-02-05T06:27:54.849410+0000","last_peered":"2024-02-05T06:27:54.849410+0000","last_clean":"2024-02-05T06:27:54.849410+0000","last_became_active":"2024-02-05T06:27:26.396996+0000","last_became_peered":"2024-02-05T06:27:26.396996+0000","last_unstale":"2024-02-05T06:27:54.849410+0000","last_undegraded":"2024-02-05T06:27:54.849410+0000","last_fullsized":"2024-02-05T06:27:54.849410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:54:35.310922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'19","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850514+0000","last_change":"2024-02-05T06:27:28.410679+0000","last_active":"2024-02-05T06:27:54.850514+0000","last_peered":"2024-02-05T06:27:54.850514+0000","last_clean":"2024-02-05T06:27:54.850514+0000","last_became_active":"2024-02-05T06:27:28.410179+0000","last_became_peered":"2024-02-05T06:27:28.410179+0000","last_unstale":"2024-02-05T06:27:54.850514+0000","last_undegraded":"2024-02-05T06:27:54.850514+0000","last_fullsized":"2024-02-05T06:27:54.850514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:07:28.997364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847312+0000","last_change":"2024-02-05T06:27:30.429578+0000","last_active":"2024-02-05T06:27:54.847312+0000","last_peered":"2024-02-05T06:27:54.847312+0000","last_clean":"2024-02-05T06:27:54.847312+0000","last_became_active":"2024-02-05T06:27:30.429157+0000","last_became_peered":"2024-02-05T06:27:30.429157+0000","last_unstale":"2024-02-05T06:27:54.847312+0000","last_undegraded":"2024-02-05T06:27:54.847312+0000","last_fullsized":"2024-02-05T06:27:54.847312+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:02:20.685813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850129+0000","last_change":"2024-02-05T06:27:32.449298+0000","last_active":"2024-02-05T06:27:54.850129+0000","last_peered":"2024-02-05T06:27:54.850129+0000","last_clean":"2024-02-05T06:27:54.850129+0000","last_became_active":"2024-02-05T06:27:32.449025+0000","last_became_peered":"2024-02-05T06:27:32.449025+0000","last_unstale":"2024-02-05T06:27:54.850129+0000","last_undegraded":"2024-02-05T06:27:54.850129+0000","last_fullsized":"2024-02-05T06:27:54.850129+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:08:23.389052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"31'18","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847995+0000","last_change":"2024-02-05T06:27:28.412028+0000","last_active":"2024-02-05T06:27:54.847995+0000","last_peered":"2024-02-05T06:27:54.847995+0000","last_clean":"2024-02-05T06:27:54.847995+0000","last_became_active":"2024-02-05T06:27:28.411744+0000","last_became_peered":"2024-02-05T06:27:28.411744+0000","last_unstale":"2024-02-05T06:27:54.847995+0000","last_undegraded":"2024-02-05T06:27:54.847995+0000","last_fullsized":"2024-02-05T06:27:54.847995+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:18:46.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847074+0000","last_change":"2024-02-05T06:27:26.396383+0000","last_active":"2024-02-05T06:27:54.847074+0000","last_peered":"2024-02-05T06:27:54.847074+0000","last_clean":"2024-02-05T06:27:54.847074+0000","last_became_active":"2024-02-05T06:27:26.395295+0000","last_became_peered":"2024-02-05T06:27:26.395295+0000","last_unstale":"2024-02-05T06:27:54.847074+0000","last_undegraded":"2024-02-05T06:27:54.847074+0000","last_fullsized":"2024-02-05T06:27:54.847074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:04:52.530255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850482+0000","last_change":"2024-02-05T06:27:30.430640+0000","last_active":"2024-02-05T06:27:54.850482+0000","last_peered":"2024-02-05T06:27:54.850482+0000","last_clean":"2024-02-05T06:27:54.850482+0000","last_became_active":"2024-02-05T06:27:30.430026+0000","last_became_peered":"2024-02-05T06:27:30.430026+0000","last_unstale":"2024-02-05T06:27:54.850482+0000","last_undegraded":"2024-02-05T06:27:54.850482+0000","last_fullsized":"2024-02-05T06:27:54.850482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:22:14.616175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848781+0000","last_change":"2024-02-05T06:27:32.445713+0000","last_active":"2024-02-05T06:27:54.848781+0000","last_peered":"2024-02-05T06:27:54.848781+0000","last_clean":"2024-02-05T06:27:54.848781+0000","last_became_active":"2024-02-05T06:27:32.445215+0000","last_became_peered":"2024-02-05T06:27:32.445215+0000","last_unstale":"2024-02-05T06:27:54.848781+0000","last_undegraded":"2024-02-05T06:27:54.848781+0000","last_fullsized":"2024-02-05T06:27:54.848781+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:32:05.241000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847471+0000","last_change":"2024-02-05T06:27:28.405805+0000","last_active":"2024-02-05T06:27:54.847471+0000","last_peered":"2024-02-05T06:27:54.847471+0000","last_clean":"2024-02-05T06:27:54.847471+0000","last_became_active":"2024-02-05T06:27:28.405476+0000","last_became_peered":"2024-02-05T06:27:28.405476+0000","last_unstale":"2024-02-05T06:27:54.847471+0000","last_undegraded":"2024-02-05T06:27:54.847471+0000","last_fullsized":"2024-02-05T06:27:54.847471+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:29:30.336402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850069+0000","last_change":"2024-02-05T06:27:26.394195+0000","last_active":"2024-02-05T06:27:54.850069+0000","last_peered":"2024-02-05T06:27:54.850069+0000","last_clean":"2024-02-05T06:27:54.850069+0000","last_became_active":"2024-02-05T06:27:26.392756+0000","last_became_peered":"2024-02-05T06:27:26.392756+0000","last_unstale":"2024-02-05T06:27:54.850069+0000","last_undegraded":"2024-02-05T06:27:54.850069+0000","last_fullsized":"2024-02-05T06:27:54.850069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:52:15.286462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"32'208","reported_seq":314,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.943168+0000","last_change":"2024-02-05T06:27:16.867788+0000","last_active":"2024-02-05T06:27:56.943168+0000","last_peered":"2024-02-05T06:27:56.943168+0000","last_clean":"2024-02-05T06:27:56.943168+0000","last_became_active":"2024-02-05T06:27:16.866776+0000","last_became_peered":"2024-02-05T06:27:16.866776+0000","last_unstale":"2024-02-05T06:27:56.943168+0000","last_undegraded":"2024-02-05T06:27:56.943168+0000","last_fullsized":"2024-02-05T06:27:56.943168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:15.855371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:15.855371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:15.855371+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:48:36.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850375+0000","last_change":"2024-02-05T06:27:30.422515+0000","last_active":"2024-02-05T06:27:54.850375+0000","last_peered":"2024-02-05T06:27:54.850375+0000","last_clean":"2024-02-05T06:27:54.850375+0000","last_became_active":"2024-02-05T06:27:30.422160+0000","last_became_peered":"2024-02-05T06:27:30.422160+0000","last_unstale":"2024-02-05T06:27:54.850375+0000","last_undegraded":"2024-02-05T06:27:54.850375+0000","last_fullsized":"2024-02-05T06:27:54.850375+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:29:54.751053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850861+0000","last_change":"2024-02-05T06:27:32.446348+0000","last_active":"2024-02-05T06:27:54.850861+0000","last_peered":"2024-02-05T06:27:54.850861+0000","last_clean":"2024-02-05T06:27:54.850861+0000","last_became_active":"2024-02-05T06:27:32.445926+0000","last_became_peered":"2024-02-05T06:27:32.445926+0000","last_unstale":"2024-02-05T06:27:54.850861+0000","last_undegraded":"2024-02-05T06:27:54.850861+0000","last_fullsized":"2024-02-05T06:27:54.850861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:25:37.424551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"31'14","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847943+0000","last_change":"2024-02-05T06:27:28.406627+0000","last_active":"2024-02-05T06:27:54.847943+0000","last_peered":"2024-02-05T06:27:54.847943+0000","last_clean":"2024-02-05T06:27:54.847943+0000","last_became_active":"2024-02-05T06:27:28.405647+0000","last_became_peered":"2024-02-05T06:27:28.405647+0000","last_unstale":"2024-02-05T06:27:54.847943+0000","last_undegraded":"2024-02-05T06:27:54.847943+0000","last_fullsized":"2024-02-05T06:27:54.847943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:50:21.174223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848771+0000","last_change":"2024-02-05T06:27:26.397456+0000","last_active":"2024-02-05T06:27:54.848771+0000","last_peered":"2024-02-05T06:27:54.848771+0000","last_clean":"2024-02-05T06:27:54.848771+0000","last_became_active":"2024-02-05T06:27:26.396961+0000","last_became_peered":"2024-02-05T06:27:26.396961+0000","last_unstale":"2024-02-05T06:27:54.848771+0000","last_undegraded":"2024-02-05T06:27:54.848771+0000","last_fullsized":"2024-02-05T06:27:54.848771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:15:40.059637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849302+0000","last_change":"2024-02-05T06:27:30.427398+0000","last_active":"2024-02-05T06:27:54.849302+0000","last_peered":"2024-02-05T06:27:54.849302+0000","last_clean":"2024-02-05T06:27:54.849302+0000","last_became_active":"2024-02-05T06:27:30.426939+0000","last_became_peered":"2024-02-05T06:27:30.426939+0000","last_unstale":"2024-02-05T06:27:54.849302+0000","last_undegraded":"2024-02-05T06:27:54.849302+0000","last_fullsized":"2024-02-05T06:27:54.849302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:30:56.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849434+0000","last_change":"2024-02-05T06:27:32.441103+0000","last_active":"2024-02-05T06:27:54.849434+0000","last_peered":"2024-02-05T06:27:54.849434+0000","last_clean":"2024-02-05T06:27:54.849434+0000","last_became_active":"2024-02-05T06:27:32.440031+0000","last_became_peered":"2024-02-05T06:27:32.440031+0000","last_unstale":"2024-02-05T06:27:54.849434+0000","last_undegraded":"2024-02-05T06:27:54.849434+0000","last_fullsized":"2024-02-05T06:27:54.849434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:52:59.139877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851222+0000","last_change":"2024-02-05T06:27:26.400277+0000","last_active":"2024-02-05T06:27:54.851222+0000","last_peered":"2024-02-05T06:27:54.851222+0000","last_clean":"2024-02-05T06:27:54.851222+0000","last_became_active":"2024-02-05T06:27:26.399980+0000","last_became_peered":"2024-02-05T06:27:26.399980+0000","last_unstale":"2024-02-05T06:27:54.851222+0000","last_undegraded":"2024-02-05T06:27:54.851222+0000","last_fullsized":"2024-02-05T06:27:54.851222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:09:02.816172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'28","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444496+0000","last_change":"2024-02-05T06:27:28.408831+0000","last_active":"2024-02-05T06:27:56.444496+0000","last_peered":"2024-02-05T06:27:56.444496+0000","last_clean":"2024-02-05T06:27:56.444496+0000","last_became_active":"2024-02-05T06:27:28.407476+0000","last_became_peered":"2024-02-05T06:27:28.407476+0000","last_unstale":"2024-02-05T06:27:56.444496+0000","last_undegraded":"2024-02-05T06:27:56.444496+0000","last_fullsized":"2024-02-05T06:27:56.444496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:37:13.422228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444488+0000","last_change":"2024-02-05T06:27:30.427766+0000","last_active":"2024-02-05T06:27:56.444488+0000","last_peered":"2024-02-05T06:27:56.444488+0000","last_clean":"2024-02-05T06:27:56.444488+0000","last_became_active":"2024-02-05T06:27:30.426677+0000","last_became_peered":"2024-02-05T06:27:30.426677+0000","last_unstale":"2024-02-05T06:27:56.444488+0000","last_undegraded":"2024-02-05T06:27:56.444488+0000","last_fullsized":"2024-02-05T06:27:56.444488+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:10:23.582817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848673+0000","last_change":"2024-02-05T06:27:32.443000+0000","last_active":"2024-02-05T06:27:54.848673+0000","last_peered":"2024-02-05T06:27:54.848673+0000","last_clean":"2024-02-05T06:27:54.848673+0000","last_became_active":"2024-02-05T06:27:32.441676+0000","last_became_peered":"2024-02-05T06:27:32.441676+0000","last_unstale":"2024-02-05T06:27:54.848673+0000","last_undegraded":"2024-02-05T06:27:54.848673+0000","last_fullsized":"2024-02-05T06:27:54.848673+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:57:40.607628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850023+0000","last_change":"2024-02-05T06:27:26.394055+0000","last_active":"2024-02-05T06:27:54.850023+0000","last_peered":"2024-02-05T06:27:54.850023+0000","last_clean":"2024-02-05T06:27:54.850023+0000","last_became_active":"2024-02-05T06:27:26.392212+0000","last_became_peered":"2024-02-05T06:27:26.392212+0000","last_unstale":"2024-02-05T06:27:54.850023+0000","last_undegraded":"2024-02-05T06:27:54.850023+0000","last_fullsized":"2024-02-05T06:27:54.850023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:22:05.181422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"31'13","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850611+0000","last_change":"2024-02-05T06:27:28.409342+0000","last_active":"2024-02-05T06:27:54.850611+0000","last_peered":"2024-02-05T06:27:54.850611+0000","last_clean":"2024-02-05T06:27:54.850611+0000","last_became_active":"2024-02-05T06:27:28.409034+0000","last_became_peered":"2024-02-05T06:27:28.409034+0000","last_unstale":"2024-02-05T06:27:54.850611+0000","last_undegraded":"2024-02-05T06:27:54.850611+0000","last_fullsized":"2024-02-05T06:27:54.850611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:30:21.100994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444478+0000","last_change":"2024-02-05T06:27:30.431185+0000","last_active":"2024-02-05T06:27:56.444478+0000","last_peered":"2024-02-05T06:27:56.444478+0000","last_clean":"2024-02-05T06:27:56.444478+0000","last_became_active":"2024-02-05T06:27:30.430696+0000","last_became_peered":"2024-02-05T06:27:30.430696+0000","last_unstale":"2024-02-05T06:27:56.444478+0000","last_undegraded":"2024-02-05T06:27:56.444478+0000","last_fullsized":"2024-02-05T06:27:56.444478+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T06:36:46.849708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849743+0000","last_change":"2024-02-05T06:27:32.445602+0000","last_active":"2024-02-05T06:27:54.849743+0000","last_peered":"2024-02-05T06:27:54.849743+0000","last_clean":"2024-02-05T06:27:54.849743+0000","last_became_active":"2024-02-05T06:27:32.445229+0000","last_became_peered":"2024-02-05T06:27:32.445229+0000","last_unstale":"2024-02-05T06:27:54.849743+0000","last_undegraded":"2024-02-05T06:27:54.849743+0000","last_fullsized":"2024-02-05T06:27:54.849743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:50:15.363905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851721+0000","last_change":"2024-02-05T06:27:26.397751+0000","last_active":"2024-02-05T06:27:54.851721+0000","last_peered":"2024-02-05T06:27:54.851721+0000","last_clean":"2024-02-05T06:27:54.851721+0000","last_became_active":"2024-02-05T06:27:26.397412+0000","last_became_peered":"2024-02-05T06:27:26.397412+0000","last_unstale":"2024-02-05T06:27:54.851721+0000","last_undegraded":"2024-02-05T06:27:54.851721+0000","last_fullsized":"2024-02-05T06:27:54.851721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:12:38.595103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"31'12","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848631+0000","last_change":"2024-02-05T06:27:28.406087+0000","last_active":"2024-02-05T06:27:54.848631+0000","last_peered":"2024-02-05T06:27:54.848631+0000","last_clean":"2024-02-05T06:27:54.848631+0000","last_became_active":"2024-02-05T06:27:28.405781+0000","last_became_peered":"2024-02-05T06:27:28.405781+0000","last_unstale":"2024-02-05T06:27:54.848631+0000","last_undegraded":"2024-02-05T06:27:54.848631+0000","last_fullsized":"2024-02-05T06:27:54.848631+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:31:31.341740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848197+0000","last_change":"2024-02-05T06:27:30.423184+0000","last_active":"2024-02-05T06:27:54.848197+0000","last_peered":"2024-02-05T06:27:54.848197+0000","last_clean":"2024-02-05T06:27:54.848197+0000","last_became_active":"2024-02-05T06:27:30.422888+0000","last_became_peered":"2024-02-05T06:27:30.422888+0000","last_unstale":"2024-02-05T06:27:54.848197+0000","last_undegraded":"2024-02-05T06:27:54.848197+0000","last_fullsized":"2024-02-05T06:27:54.848197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:26:21.047318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851086+0000","last_change":"2024-02-05T06:27:32.446254+0000","last_active":"2024-02-05T06:27:54.851086+0000","last_peered":"2024-02-05T06:27:54.851086+0000","last_clean":"2024-02-05T06:27:54.851086+0000","last_became_active":"2024-02-05T06:27:32.445736+0000","last_became_peered":"2024-02-05T06:27:32.445736+0000","last_unstale":"2024-02-05T06:27:54.851086+0000","last_undegraded":"2024-02-05T06:27:54.851086+0000","last_fullsized":"2024-02-05T06:27:54.851086+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:22:34.004152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444587+0000","last_change":"2024-02-05T06:27:30.422795+0000","last_active":"2024-02-05T06:27:56.444587+0000","last_peered":"2024-02-05T06:27:56.444587+0000","last_clean":"2024-02-05T06:27:56.444587+0000","last_became_active":"2024-02-05T06:27:30.422307+0000","last_became_peered":"2024-02-05T06:27:30.422307+0000","last_unstale":"2024-02-05T06:27:56.444587+0000","last_undegraded":"2024-02-05T06:27:56.444587+0000","last_fullsized":"2024-02-05T06:27:56.444587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:22.186920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848834+0000","last_change":"2024-02-05T06:27:26.399213+0000","last_active":"2024-02-05T06:27:54.848834+0000","last_peered":"2024-02-05T06:27:54.848834+0000","last_clean":"2024-02-05T06:27:54.848834+0000","last_became_active":"2024-02-05T06:27:26.398585+0000","last_became_peered":"2024-02-05T06:27:26.398585+0000","last_unstale":"2024-02-05T06:27:54.848834+0000","last_undegraded":"2024-02-05T06:27:54.848834+0000","last_fullsized":"2024-02-05T06:27:54.848834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:19:26.201195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851047+0000","last_change":"2024-02-05T06:27:28.412084+0000","last_active":"2024-02-05T06:27:54.851047+0000","last_peered":"2024-02-05T06:27:54.851047+0000","last_clean":"2024-02-05T06:27:54.851047+0000","last_became_active":"2024-02-05T06:27:28.411769+0000","last_became_peered":"2024-02-05T06:27:28.411769+0000","last_unstale":"2024-02-05T06:27:54.851047+0000","last_undegraded":"2024-02-05T06:27:54.851047+0000","last_fullsized":"2024-02-05T06:27:54.851047+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:50:47.398036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849673+0000","last_change":"2024-02-05T06:27:32.445521+0000","last_active":"2024-02-05T06:27:54.849673+0000","last_peered":"2024-02-05T06:27:54.849673+0000","last_clean":"2024-02-05T06:27:54.849673+0000","last_became_active":"2024-02-05T06:27:32.445017+0000","last_became_peered":"2024-02-05T06:27:32.445017+0000","last_unstale":"2024-02-05T06:27:54.849673+0000","last_undegraded":"2024-02-05T06:27:54.849673+0000","last_fullsized":"2024-02-05T06:27:54.849673+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:17:00.166203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850146+0000","last_change":"2024-02-05T06:27:30.430250+0000","last_active":"2024-02-05T06:27:54.850146+0000","last_peered":"2024-02-05T06:27:54.850146+0000","last_clean":"2024-02-05T06:27:54.850146+0000","last_became_active":"2024-02-05T06:27:30.429003+0000","last_became_peered":"2024-02-05T06:27:30.429003+0000","last_unstale":"2024-02-05T06:27:54.850146+0000","last_undegraded":"2024-02-05T06:27:54.850146+0000","last_fullsized":"2024-02-05T06:27:54.850146+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:20:11.672144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847425+0000","last_change":"2024-02-05T06:27:26.395355+0000","last_active":"2024-02-05T06:27:54.847425+0000","last_peered":"2024-02-05T06:27:54.847425+0000","last_clean":"2024-02-05T06:27:54.847425+0000","last_became_active":"2024-02-05T06:27:26.394700+0000","last_became_peered":"2024-02-05T06:27:26.394700+0000","last_unstale":"2024-02-05T06:27:54.847425+0000","last_undegraded":"2024-02-05T06:27:54.847425+0000","last_fullsized":"2024-02-05T06:27:54.847425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:14:47.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848617+0000","last_change":"2024-02-05T06:27:28.412358+0000","last_active":"2024-02-05T06:27:54.848617+0000","last_peered":"2024-02-05T06:27:54.848617+0000","last_clean":"2024-02-05T06:27:54.848617+0000","last_became_active":"2024-02-05T06:27:28.411707+0000","last_became_peered":"2024-02-05T06:27:28.411707+0000","last_unstale":"2024-02-05T06:27:54.848617+0000","last_undegraded":"2024-02-05T06:27:54.848617+0000","last_fullsized":"2024-02-05T06:27:54.848617+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:23:02.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848848+0000","last_change":"2024-02-05T06:27:32.446030+0000","last_active":"2024-02-05T06:27:54.848848+0000","last_peered":"2024-02-05T06:27:54.848848+0000","last_clean":"2024-02-05T06:27:54.848848+0000","last_became_active":"2024-02-05T06:27:32.445738+0000","last_became_peered":"2024-02-05T06:27:32.445738+0000","last_unstale":"2024-02-05T06:27:54.848848+0000","last_undegraded":"2024-02-05T06:27:54.848848+0000","last_fullsized":"2024-02-05T06:27:54.848848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:09:14.377854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848971+0000","last_change":"2024-02-05T06:27:26.398020+0000","last_active":"2024-02-05T06:27:54.848971+0000","last_peered":"2024-02-05T06:27:54.848971+0000","last_clean":"2024-02-05T06:27:54.848971+0000","last_became_active":"2024-02-05T06:27:26.397640+0000","last_became_peered":"2024-02-05T06:27:26.397640+0000","last_unstale":"2024-02-05T06:27:54.848971+0000","last_undegraded":"2024-02-05T06:27:54.848971+0000","last_fullsized":"2024-02-05T06:27:54.848971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:26:43.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850137+0000","last_change":"2024-02-05T06:27:28.405872+0000","last_active":"2024-02-05T06:27:54.850137+0000","last_peered":"2024-02-05T06:27:54.850137+0000","last_clean":"2024-02-05T06:27:54.850137+0000","last_became_active":"2024-02-05T06:27:28.405538+0000","last_became_peered":"2024-02-05T06:27:28.405538+0000","last_unstale":"2024-02-05T06:27:54.850137+0000","last_undegraded":"2024-02-05T06:27:54.850137+0000","last_fullsized":"2024-02-05T06:27:54.850137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:15:30.126689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850254+0000","last_change":"2024-02-05T06:27:30.424420+0000","last_active":"2024-02-05T06:27:54.850254+0000","last_peered":"2024-02-05T06:27:54.850254+0000","last_clean":"2024-02-05T06:27:54.850254+0000","last_became_active":"2024-02-05T06:27:30.424016+0000","last_became_peered":"2024-02-05T06:27:30.424016+0000","last_unstale":"2024-02-05T06:27:54.850254+0000","last_undegraded":"2024-02-05T06:27:54.850254+0000","last_fullsized":"2024-02-05T06:27:54.850254+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:45:24.882993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847078+0000","last_change":"2024-02-05T06:27:32.446083+0000","last_active":"2024-02-05T06:27:54.847078+0000","last_peered":"2024-02-05T06:27:54.847078+0000","last_clean":"2024-02-05T06:27:54.847078+0000","last_became_active":"2024-02-05T06:27:32.445450+0000","last_became_peered":"2024-02-05T06:27:32.445450+0000","last_unstale":"2024-02-05T06:27:54.847078+0000","last_undegraded":"2024-02-05T06:27:54.847078+0000","last_fullsized":"2024-02-05T06:27:54.847078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:02:10.288112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'8","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:56.444592+0000","last_change":"2024-02-05T06:27:30.422801+0000","last_active":"2024-02-05T06:27:56.444592+0000","last_peered":"2024-02-05T06:27:56.444592+0000","last_clean":"2024-02-05T06:27:56.444592+0000","last_became_active":"2024-02-05T06:27:30.422395+0000","last_became_peered":"2024-02-05T06:27:30.422395+0000","last_unstale":"2024-02-05T06:27:56.444592+0000","last_undegraded":"2024-02-05T06:27:56.444592+0000","last_fullsized":"2024-02-05T06:27:56.444592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:53:10.663620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849402+0000","last_change":"2024-02-05T06:27:26.398022+0000","last_active":"2024-02-05T06:27:54.849402+0000","last_peered":"2024-02-05T06:27:54.849402+0000","last_clean":"2024-02-05T06:27:54.849402+0000","last_became_active":"2024-02-05T06:27:26.397692+0000","last_became_peered":"2024-02-05T06:27:26.397692+0000","last_unstale":"2024-02-05T06:27:54.849402+0000","last_undegraded":"2024-02-05T06:27:54.849402+0000","last_fullsized":"2024-02-05T06:27:54.849402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:19:26.852320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"31'19","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849570+0000","last_change":"2024-02-05T06:27:28.409122+0000","last_active":"2024-02-05T06:27:54.849570+0000","last_peered":"2024-02-05T06:27:54.849570+0000","last_clean":"2024-02-05T06:27:54.849570+0000","last_became_active":"2024-02-05T06:27:28.408759+0000","last_became_peered":"2024-02-05T06:27:28.408759+0000","last_unstale":"2024-02-05T06:27:54.849570+0000","last_undegraded":"2024-02-05T06:27:54.849570+0000","last_fullsized":"2024-02-05T06:27:54.849570+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:57:15.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848408+0000","last_change":"2024-02-05T06:27:32.445655+0000","last_active":"2024-02-05T06:27:54.848408+0000","last_peered":"2024-02-05T06:27:54.848408+0000","last_clean":"2024-02-05T06:27:54.848408+0000","last_became_active":"2024-02-05T06:27:32.445349+0000","last_became_peered":"2024-02-05T06:27:32.445349+0000","last_unstale":"2024-02-05T06:27:54.848408+0000","last_undegraded":"2024-02-05T06:27:54.848408+0000","last_fullsized":"2024-02-05T06:27:54.848408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:47:19.903620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850007+0000","last_change":"2024-02-05T06:27:26.399604+0000","last_active":"2024-02-05T06:27:54.850007+0000","last_peered":"2024-02-05T06:27:54.850007+0000","last_clean":"2024-02-05T06:27:54.850007+0000","last_became_active":"2024-02-05T06:27:26.399145+0000","last_became_peered":"2024-02-05T06:27:26.399145+0000","last_unstale":"2024-02-05T06:27:54.850007+0000","last_undegraded":"2024-02-05T06:27:54.850007+0000","last_fullsized":"2024-02-05T06:27:54.850007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:49:34.114145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"31'17","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850654+0000","last_change":"2024-02-05T06:27:28.410240+0000","last_active":"2024-02-05T06:27:54.850654+0000","last_peered":"2024-02-05T06:27:54.850654+0000","last_clean":"2024-02-05T06:27:54.850654+0000","last_became_active":"2024-02-05T06:27:28.409858+0000","last_became_peered":"2024-02-05T06:27:28.409858+0000","last_unstale":"2024-02-05T06:27:54.850654+0000","last_undegraded":"2024-02-05T06:27:54.850654+0000","last_fullsized":"2024-02-05T06:27:54.850654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:50:31.223880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849290+0000","last_change":"2024-02-05T06:27:30.425854+0000","last_active":"2024-02-05T06:27:54.849290+0000","last_peered":"2024-02-05T06:27:54.849290+0000","last_clean":"2024-02-05T06:27:54.849290+0000","last_became_active":"2024-02-05T06:27:30.424265+0000","last_became_peered":"2024-02-05T06:27:30.424265+0000","last_unstale":"2024-02-05T06:27:54.849290+0000","last_undegraded":"2024-02-05T06:27:54.849290+0000","last_fullsized":"2024-02-05T06:27:54.849290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:59:52.233110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847000+0000","last_change":"2024-02-05T06:27:32.445271+0000","last_active":"2024-02-05T06:27:54.847000+0000","last_peered":"2024-02-05T06:27:54.847000+0000","last_clean":"2024-02-05T06:27:54.847000+0000","last_became_active":"2024-02-05T06:27:32.444920+0000","last_became_peered":"2024-02-05T06:27:32.444920+0000","last_unstale":"2024-02-05T06:27:54.847000+0000","last_undegraded":"2024-02-05T06:27:54.847000+0000","last_fullsized":"2024-02-05T06:27:54.847000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:26:18.241336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849896+0000","last_change":"2024-02-05T06:27:26.399446+0000","last_active":"2024-02-05T06:27:54.849896+0000","last_peered":"2024-02-05T06:27:54.849896+0000","last_clean":"2024-02-05T06:27:54.849896+0000","last_became_active":"2024-02-05T06:27:26.399162+0000","last_became_peered":"2024-02-05T06:27:26.399162+0000","last_unstale":"2024-02-05T06:27:54.849896+0000","last_undegraded":"2024-02-05T06:27:54.849896+0000","last_fullsized":"2024-02-05T06:27:54.849896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:36:23.180118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850624+0000","last_change":"2024-02-05T06:27:28.410597+0000","last_active":"2024-02-05T06:27:54.850624+0000","last_peered":"2024-02-05T06:27:54.850624+0000","last_clean":"2024-02-05T06:27:54.850624+0000","last_became_active":"2024-02-05T06:27:28.410011+0000","last_became_peered":"2024-02-05T06:27:28.410011+0000","last_unstale":"2024-02-05T06:27:54.850624+0000","last_undegraded":"2024-02-05T06:27:54.850624+0000","last_fullsized":"2024-02-05T06:27:54.850624+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:11:02.504562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.847714+0000","last_change":"2024-02-05T06:27:30.430793+0000","last_active":"2024-02-05T06:27:54.847714+0000","last_peered":"2024-02-05T06:27:54.847714+0000","last_clean":"2024-02-05T06:27:54.847714+0000","last_became_active":"2024-02-05T06:27:30.430134+0000","last_became_peered":"2024-02-05T06:27:30.430134+0000","last_unstale":"2024-02-05T06:27:54.847714+0000","last_undegraded":"2024-02-05T06:27:54.847714+0000","last_fullsized":"2024-02-05T06:27:54.847714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:08:57.287272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.848787+0000","last_change":"2024-02-05T06:27:32.442752+0000","last_active":"2024-02-05T06:27:54.848787+0000","last_peered":"2024-02-05T06:27:54.848787+0000","last_clean":"2024-02-05T06:27:54.848787+0000","last_became_active":"2024-02-05T06:27:32.441194+0000","last_became_peered":"2024-02-05T06:27:32.441194+0000","last_unstale":"2024-02-05T06:27:54.848787+0000","last_undegraded":"2024-02-05T06:27:54.848787+0000","last_fullsized":"2024-02-05T06:27:54.848787+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:31.421195+0000","last_clean_scrub_stamp":"2024-02-05T06:27:31.421195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:01:37.958119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.850282+0000","last_change":"2024-02-05T06:27:26.397248+0000","last_active":"2024-02-05T06:27:54.850282+0000","last_peered":"2024-02-05T06:27:54.850282+0000","last_clean":"2024-02-05T06:27:54.850282+0000","last_became_active":"2024-02-05T06:27:26.395102+0000","last_became_peered":"2024-02-05T06:27:26.395102+0000","last_unstale":"2024-02-05T06:27:54.850282+0000","last_undegraded":"2024-02-05T06:27:54.850282+0000","last_fullsized":"2024-02-05T06:27:54.850282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:25.380803+0000","last_clean_scrub_stamp":"2024-02-05T06:27:25.380803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:53:27.344825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.851732+0000","last_change":"2024-02-05T06:27:28.411647+0000","last_active":"2024-02-05T06:27:54.851732+0000","last_peered":"2024-02-05T06:27:54.851732+0000","last_clean":"2024-02-05T06:27:54.851732+0000","last_became_active":"2024-02-05T06:27:28.411113+0000","last_became_peered":"2024-02-05T06:27:28.411113+0000","last_unstale":"2024-02-05T06:27:54.851732+0000","last_undegraded":"2024-02-05T06:27:54.851732+0000","last_fullsized":"2024-02-05T06:27:54.851732+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:27.392371+0000","last_clean_scrub_stamp":"2024-02-05T06:27:27.392371+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:05:05.481584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-05T06:27:54.849288+0000","last_change":"2024-02-05T06:27:30.430446+0000","last_active":"2024-02-05T06:27:54.849288+0000","last_peered":"2024-02-05T06:27:54.849288+0000","last_clean":"2024-02-05T06:27:54.849288+0000","last_became_active":"2024-02-05T06:27:30.430109+0000","last_became_peered":"2024-02-05T06:27:30.430109+0000","last_unstale":"2024-02-05T06:27:54.849288+0000","last_undegraded":"2024-02-05T06:27:54.849288+0000","last_fullsized":"2024-02-05T06:27:54.849288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T06:27:29.405739+0000","last_clean_scrub_stamp":"2024-02-05T06:27:29.405739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:13:22.266437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708840,"statfs":{"total":95990841344,"available":95957852160,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 06:27:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 06:27:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T06:28:12.234 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-05T06:28:12.234 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-05T06:28:12.234 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-05T06:28:12.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-05T06:28:12.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:12 smithi142 ceph-mon[30053]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:12.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:12 smithi122 ceph-mon[25512]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:12.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:12 smithi196 ceph-mon[30516]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:13.093 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:14.503 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:14 smithi122 ceph-mon[25512]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:14.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:14 smithi142 ceph-mon[30053]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:14.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:14 smithi196 ceph-mon[30516]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-05T06:28:16.529 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:16 smithi122 ceph-mon[25512]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:16.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:16 smithi142 ceph-mon[30053]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:16.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:16 smithi196 ceph-mon[30516]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:16.905 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-02-05T06:28:16.905 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-05T06:28:17.143 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:16 smithi122 podman[54722]: 2024-02-05 06:28:16.942690348 +0000 UTC m=+0.214827630 container create 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:17.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:17 smithi142 ceph-mon[30053]: from='client.? 172.21.15.122:0/2784415073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T06:28:17.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-mon[25512]: from='client.? 172.21.15.122:0/2784415073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T06:28:17.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:17 smithi196 ceph-mon[30516]: from='client.? 172.21.15.122:0/2784415073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T06:28:18.129 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 podman[54722]: 2024-02-05 06:28:17.874196358 +0000 UTC m=+1.146333648 container init 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 podman[54722]: 2024-02-05 06:28:17.88120928 +0000 UTC m=+1.153346572 container start 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.896Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.896Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.898Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.122 port=9094 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.899Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.923Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.924Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.926Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:17.926Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 bash[54722]: 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 2024-02-05T06:28:18.130 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:17 smithi122 systemd[1]: Started Ceph alertmanager.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE Bus STOPPING 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE Bus STOPPED 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE Bus STARTING 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE Serving on http://:::9283 2024-02-05T06:28:18.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:18 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:18] ENGINE Bus STARTED 2024-02-05T06:28:18.415 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-05T06:28:18.415 INFO:tasks.cephadm:Setup complete, yielding 2024-02-05T06:28:18.416 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-05T06:28:18.425 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-02-05T06:28:18.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:18.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:18 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:18.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:18 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:19.307 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ::ffff:172.21.15.142 - - [05/Feb/2024:06:28:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T06:28:19.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:19 smithi142 ceph-mon[30053]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T06:28:19.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:19 smithi142 ceph-mon[30053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:19.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:19 smithi142 ceph-mon[30053]: Deploying daemon grafana.a on smithi142 2024-02-05T06:28:19.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:19 smithi122 ceph-mon[25512]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T06:28:19.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:19 smithi122 ceph-mon[25512]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:19.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:19 smithi122 ceph-mon[25512]: Deploying daemon grafana.a on smithi142 2024-02-05T06:28:19.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:19 smithi196 ceph-mon[30516]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T06:28:19.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:19 smithi196 ceph-mon[30516]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T06:28:19.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:19 smithi196 ceph-mon[30516]: Deploying daemon grafana.a on smithi142 2024-02-05T06:28:20.036 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:19 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:19.900Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001039063s 2024-02-05T06:28:20.249 INFO:teuthology.orchestra.run.smithi122.stdout:167 167 2024-02-05T06:28:20.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:20 smithi142 ceph-mon[30053]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:20.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:20 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:20.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:20 smithi122 ceph-mon[25512]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:20.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:20 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:20.771 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:20 smithi196 ceph-mon[30516]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:20.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:20 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:20.882 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch status' 2024-02-05T06:28:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:22.413 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:22 smithi122 ceph-mon[25512]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:22.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:22 smithi196 ceph-mon[30516]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:22.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:22 smithi142 ceph-mon[30053]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:22.852 INFO:teuthology.orchestra.run.smithi122.stdout:Backend: cephadm 2024-02-05T06:28:22.852 INFO:teuthology.orchestra.run.smithi122.stdout:Available: Yes 2024-02-05T06:28:22.852 INFO:teuthology.orchestra.run.smithi122.stdout:Paused: No 2024-02-05T06:28:23.529 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-05T06:28:23.764 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:24.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:24 smithi122 ceph-mon[25512]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:24.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:24 smithi122 ceph-mon[25512]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:24.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:24 smithi196 ceph-mon[30516]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:24.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:24 smithi196 ceph-mon[30516]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:24.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:24 smithi142 ceph-mon[30053]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:24.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:24 smithi142 ceph-mon[30053]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:25.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:25 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:25.574 INFO:teuthology.orchestra.run.smithi122.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:alertmanager.a smithi122 *:9093,9094 starting - - - - 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.x smithi142 *:8443,8765 running (2m) 27s ago 2m 447M - 19.0.0-1241-g164d3be4 8f4bf3e8b16a 80e1952f7881 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.y smithi122 *:9283,8765 running (4m) 21s ago 4m 483M - 19.0.0-1241-g164d3be4 8f4bf3e8b16a 9bd3bd6892db 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.z smithi196 *:8443,8765 running (2m) 28s ago 2m 447M - 19.0.0-1241-g164d3be4 8f4bf3e8b16a 1c0ae5d7332e 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mon.a smithi122 running (4m) 21s ago 4m 49.7M 2048M 19.0.0-1241-g164d3be4 8f4bf3e8b16a b1b1301aec25 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mon.b smithi142 running (2m) 27s ago 2m 42.9M 2048M 19.0.0-1241-g164d3be4 8f4bf3e8b16a d2eed94ac3c1 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:mon.c smithi196 running (2m) 28s ago 2m 42.9M 2048M 19.0.0-1241-g164d3be4 8f4bf3e8b16a cbd9c0063861 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 smithi122 running (2m) 21s ago 2m 51.8M 15.6G 19.0.0-1241-g164d3be4 8f4bf3e8b16a f30e829374ac 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 smithi142 running (102s) 27s ago 102s 52.9M 15.6G 19.0.0-1241-g164d3be4 8f4bf3e8b16a aac4633c4da2 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 smithi196 running (78s) 28s ago 78s 51.0M 16.6G 19.0.0-1241-g164d3be4 8f4bf3e8b16a 2b895baf2627 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:prometheus.a smithi142 *:9095 running (39s) 27s ago 38s 26.0M - 2.43.0 a07b618ecd1d 2e3df5e22d48 2024-02-05T06:28:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:rgw.foo.a smithi122 *:80 running (60s) 21s ago 60s 78.1M - 19.0.0-1241-g164d3be4 8f4bf3e8b16a 99f77f441ffe 2024-02-05T06:28:25.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:25 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:25.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:25 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:26.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-05T06:28:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:26.499 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:26 smithi122 ceph-mon[25512]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:26.499 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:26 smithi122 ceph-mon[25512]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:26.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:26 smithi196 ceph-mon[30516]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:26.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:26 smithi196 ceph-mon[30516]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:26.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:26 smithi142 ceph-mon[30053]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:26.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:26 smithi142 ceph-mon[30053]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:28.286 INFO:teuthology.orchestra.run.smithi122.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-05T06:28:28.286 INFO:teuthology.orchestra.run.smithi122.stdout:alertmanager ?:9093,9094 0/1 - 58s smithi122=a;count:1 2024-02-05T06:28:28.286 INFO:teuthology.orchestra.run.smithi122.stdout:grafana ?:3000 0/1 - 55s smithi142=a;count:1 2024-02-05T06:28:28.286 INFO:teuthology.orchestra.run.smithi122.stdout:mgr 3/3 31s ago 2m smithi122=y;smithi142=x;smithi196=z;count:3 2024-02-05T06:28:28.287 INFO:teuthology.orchestra.run.smithi122.stdout:mon 3/3 31s ago 2m smithi122:172.21.15.122=a;smithi142:172.21.15.142=b;smithi196:172.21.15.196=c;count:3 2024-02-05T06:28:28.287 INFO:teuthology.orchestra.run.smithi122.stdout:node-exporter ?:9100 0/3 - 61s smithi122=a;smithi142=b;smithi196=c;count:3 2024-02-05T06:28:28.287 INFO:teuthology.orchestra.run.smithi122.stdout:osd 3 31s ago - 2024-02-05T06:28:28.287 INFO:teuthology.orchestra.run.smithi122.stdout:prometheus ?:9095 1/1 30s ago 63s smithi142=a;count:1 2024-02-05T06:28:28.287 INFO:teuthology.orchestra.run.smithi122.stdout:rgw.foo.a ?:80 1/1 24s ago 63s smithi122=foo.a;count:1 2024-02-05T06:28:28.287 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:28:27 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:28:27.902Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002971387s 2024-02-05T06:28:28.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:28 smithi122 ceph-mon[25512]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:28.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:28 smithi196 ceph-mon[30516]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:28.812 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:28 smithi142 ceph-mon[30053]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:28.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-05T06:28:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:29.451 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:29 smithi122 ceph-mon[25512]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:29.451 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:29 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ::ffff:172.21.15.142 - - [05/Feb/2024:06:28:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T06:28:29.673 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:29 smithi196 ceph-mon[30516]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:29.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:29 smithi142 ceph-mon[30053]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:30.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:30 smithi122 ceph-mon[25512]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:30.636 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:30 smithi142 ceph-mon[30053]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:30.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:30 smithi196 ceph-mon[30516]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:31.107 INFO:teuthology.orchestra.run.smithi122.stdout:HOST ADDR LABELS STATUS 2024-02-05T06:28:31.107 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 172.21.15.122 2024-02-05T06:28:31.107 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 172.21.15.142 2024-02-05T06:28:31.107 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 172.21.15.196 2024-02-05T06:28:31.107 INFO:teuthology.orchestra.run.smithi122.stdout:3 hosts in cluster 2024-02-05T06:28:31.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-05T06:28:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:32.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 systemd[1]: Starting Ceph grafana.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:32.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-mon[30053]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:32.562 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-mon[30053]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:32.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 podman[42297]: 2024-02-05 06:28:32.299243989 +0000 UTC m=+0.114640267 container create 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, io.buildah.version=1.32.0, vcs-type=git, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, release=239.1651231664, distribution-scope=public, vendor=Red Hat, Inc., architecture=x86_64, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-05T06:28:32.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 podman[42297]: 2024-02-05 06:28:32.201745985 +0000 UTC m=+0.017142265 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-05T06:28:32.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:32 smithi122 ceph-mon[25512]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:32.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:32 smithi122 ceph-mon[25512]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:32.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:32 smithi196 ceph-mon[30516]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:32.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:32 smithi196 ceph-mon[30516]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 podman[42297]: 2024-02-05 06:28:32.593780293 +0000 UTC m=+0.409176580 container init 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, release=239.1651231664, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, maintainer=Paul Cuzner , distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, description=Ceph Grafana Container, io.openshift.tags=base rhel8) 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 podman[42297]: 2024-02-05 06:28:32.599150033 +0000 UTC m=+0.414546320 container start 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, build-date=2022-04-29T11:48:11.055897, version=8.5, io.openshift.expose-services=, description=Ceph Grafana Container, architecture=x86_64, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, release=239.1651231664, vcs-type=git, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 bash[42297]: 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 systemd[1]: Started Ceph grafana.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757442858Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757642412Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757660629Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757669217Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-05T06:28:32.840 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757677104Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757683499Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757689255Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757694908Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757700802Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757706313Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757711957Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757717303Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757722903Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=settings t=2024-02-05T06:28:32.757728802Z level=info msg="App mode production" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore t=2024-02-05T06:28:32.757770177Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore t=2024-02-05T06:28:32.757786043Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.758782158Z level=info msg="Starting DB migrations" 2024-02-05T06:28:32.841 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.758856827Z level=info msg="Executing migration" id="create migration_log table" 2024-02-05T06:28:32.842 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.759385593Z level=info msg="Executing migration" id="create user table" 2024-02-05T06:28:32.842 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.759811367Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-05T06:28:32.842 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.760291269Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-05T06:28:32.842 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.760794492Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-05T06:28:32.842 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.761251791Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.761649945Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.762350675Z level=info msg="Executing migration" id="create user table v2" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.762758661Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.763175041Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.763605206Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.763939062Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.76429834Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.764731815Z level=info msg="Executing migration" id="Update user table charset" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.764995992Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-05T06:28:32.843 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.765409664Z level=info msg="Executing migration" id="Add missing user data" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.765729485Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.766148238Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.766536053Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.766924859Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.768010773Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.768447479Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.768873472Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.769288526Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.770127816Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.770992348Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.771797302Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.77267429Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.773518054Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-05T06:28:32.844 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.774346045Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.775169541Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.776028807Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.776864516Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.777294773Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.778094036Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.778492813Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.778990296Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.781737746Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.784568119Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.787446641Z level=info msg="Executing migration" id="create star table" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.790201593Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.795691202Z level=info msg="Executing migration" id="create org table v1" 2024-02-05T06:28:32.845 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.796393969Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.796817007Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.799308139Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.799740083Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.802368267Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.804913185Z level=info msg="Executing migration" id="Update org table charset" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.807459973Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.810017779Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.812560511Z level=info msg="Executing migration" id="create dashboard table" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.815086571Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.815953936Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.816401707Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.81729038Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-05T06:28:32.846 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.818025721Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.818907496Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.819771181Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.820625754Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.821494651Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.822345009Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.82269274Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.823563373Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.824410243Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.825267104Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.825756021Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.826625563Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.827443572Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-05T06:28:32.847 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.828299712Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.82873662Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.829600693Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.830485467Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.831364409Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.831861591Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.832706703Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.833609591Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.834487936Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.834817716Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.835661479Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.836530972Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.83741055Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.837824953Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-05T06:28:32.848 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.838682531Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.839555182Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.840438748Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.840856254Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.841712364Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.844436605Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.844912756Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-05T06:28:32.849 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.847222561Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-05T06:28:33.092 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.847693084Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.850496907Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.850825505Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.853260118Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.856184722Z level=info msg="Executing migration" id="create data_source table" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.856622195Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.85949705Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.862261137Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.865094663Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.865502873Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.868286294Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.871020286Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-05T06:28:33.093 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.87389018Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.874315434Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.876991487Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.877560274Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.879524966Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.882167809Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-05T06:28:33.094 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.882491664Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.88516979Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.885498732Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.887690399Z level=info msg="Executing migration" id="Add uid column" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.888232296Z level=info msg="Executing migration" id="Update uid value" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.890362517Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.89079255Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.893358893Z level=info msg="Executing migration" id="create api_key table" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.895959312Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-05T06:28:33.095 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.898597461Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.899038Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.901605128Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.904226737Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.906794931Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.907581608Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.908621977Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.909516634Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.910351988Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.911230244Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.91207562Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.912923665Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-05T06:28:33.096 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.913770092Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.914620068Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.91547983Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.916433298Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.918335963Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.919125368Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.91998078Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.920822589Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.921679388Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.922532773Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.923395755Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.924288722Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.925130658Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.925974853Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.926865548Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-05T06:28:33.097 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.927713664Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.928618377Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.929513457Z level=info msg="Executing migration" id="create quota table v1" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.930364177Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.931208928Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.932055693Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.932942519Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.933824898Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.934705068Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.93553139Z level=info msg="Executing migration" id="create session table" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.936422956Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.937314703Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.938114962Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-05T06:28:33.098 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.940882265Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.943788138Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.946737303Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.949514904Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.952379878Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.955328705Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.958018906Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.960954987Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.963860963Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.966731159Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.969520263Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.972354293Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.975171257Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.978028967Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-05T06:28:33.099 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.980846843Z level=info msg="Executing migration" id="create alert table v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.983543117Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.986325726Z level=info msg="Executing migration" id="add index alert state" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.986749839Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.987174001Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.987549266Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.987970153Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.98840827Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.991257625Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.991686971Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.9921158Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.992434526Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.992768032Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.994556188Z level=info msg="Executing migration" id="Add column is_default" 2024-02-05T06:28:33.100 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.997279972Z level=info msg="Executing migration" id="Add column frequency" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.997954132Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.998599558Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.999230466Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.999652554Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:32 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:32.999932356Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.0002499Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.000628774Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.001066698Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.001455391Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.004107867Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.004545141Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.006985358Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.009780718Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.012499897Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-05T06:28:33.101 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.01526032Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-05T06:28:33.102 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.017962732Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-05T06:28:33.102 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.02077558Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-05T06:28:33.102 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.023358152Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-05T06:28:33.102 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.025806924Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.028495941Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.031009737Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.033796295Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.036372641Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.039111588Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.041717029Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.044175779Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.046880079Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.049516085Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.052187601Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.054946902Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.057593354Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-05T06:28:33.103 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.058432403Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.059250296Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.060098246Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.061659199Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.06250382Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.063358389Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.064168036Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.06501046Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.065889485Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.066679129Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.067546985Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.068423806Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.069304219Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.070128034Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-05T06:28:33.104 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.070989709Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.071841748Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.072660914Z level=info msg="Executing migration" id="Move region to single row" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.073543144Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.074405545Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.075295846Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.076153661Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.077037937Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.077924789Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.078766148Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.079621404Z level=info msg="Executing migration" id="create test_data table" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.080409679Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.080883935Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.081704261Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-05T06:28:33.105 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.082584649Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.083432125Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.084265436Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.085118692Z level=info msg="Executing migration" id="create team table" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.085982576Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.086836154Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.087708039Z level=info msg="Executing migration" id="create team member table" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.088555878Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.091383453Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.094128245Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.097177052Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-05T06:28:33.106 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.099929486Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-05T06:28:33.344 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.102639482Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-05T06:28:33.344 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.105419618Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-05T06:28:33.344 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.108276358Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-05T06:28:33.344 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.111108314Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-05T06:28:33.344 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.113884989Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.114332611Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.117012596Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.117430011Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.117834133Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.118265237Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.118656012Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.118938484Z level=info msg="Executing migration" id="create tag table" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.119310599Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.119715335Z level=info msg="Executing migration" id="create login attempt table" 2024-02-05T06:28:33.345 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.120095366Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.120505435Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.120870821Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.122803062Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.123405528Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.123811772Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.12412951Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.12449335Z level=info msg="Executing migration" id="create user auth table" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.124839188Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.125303953Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.125587185Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.126329729Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.12705976Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.127771514Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-05T06:28:33.346 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.1285054Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.128925116Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.129654329Z level=info msg="Executing migration" id="create server_lock table" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.130037265Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.130466189Z level=info msg="Executing migration" id="create user auth token table" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.130872249Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.131290795Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.131689609Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.13210062Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-05T06:28:33.347 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.132864044Z level=info msg="Executing migration" id="create cache_data table" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.133314034Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.133751109Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.134184034Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.134617768Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.134895836Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.135213049Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.136997729Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-05T06:28:33.349 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.139894699Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.142571511Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.142892961Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.143339397Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.146136624Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.146601148Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.148896559Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.14972941Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.152646539Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.155419511Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.158168256Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.158621023Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.161404496Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-05T06:28:33.350 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.164025375Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.166734609Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.167172844Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.169752001Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.172446043Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.175031681Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.175492176Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.178146649Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.181569867Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.184986162Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.187547757Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.187989949Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-05T06:28:33.351 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.190615539Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.193121484Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.195644442Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.196104366Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.198665668Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.199510877Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.200388698Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.201197528Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.20198762Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.202829737Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.203697393Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.20454871Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.205428132Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-05T06:28:33.352 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.206291553Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.207098664Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.207917404Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.208727096Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.209587243Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.21043472Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.210850721Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.211719135Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.212606631Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.213481655Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.214329843Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.215191367Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.216066501Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-05T06:28:33.353 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.216919997Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.217311791Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.218181649Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.219061701Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.219970025Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.2204186Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.221312054Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.222162923Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.223006475Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.223843512Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.224711304Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.22559374Z level=info msg="Executing migration" id="create alert_image table" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.226437808Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-05T06:28:33.354 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.226859839Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.229593314Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.232404417Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.235343529Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.235419786Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.235688797Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.238591501Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.241312792Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.241428429Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.241548931Z level=info msg="alerts found to migrate" alerts=0 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.244057857Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.24453475Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.24742386Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-05T06:28:33.355 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.250157047Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.252914575Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.253413991Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.256256442Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.256712991Z level=info msg="Executing migration" id="create data_keys table" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.259179023Z level=info msg="Executing migration" id="create secrets table" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.261876943Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.26620089Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.268944257Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.271646012Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.275992113Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.280194996Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.282792119Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-05T06:28:33.356 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.285617552Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.288234923Z level=info msg="Executing migration" id="create permission table" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.288647396Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.291403972Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.294031686Z level=info msg="Executing migration" id="create role table" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.296652967Z level=info msg="Executing migration" id="add column display_name" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.297707273Z level=info msg="Executing migration" id="add column group_name" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.300376767Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.303023656Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.30570341Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.306310837Z level=info msg="Executing migration" id="create team role table" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.309182418Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.311839374Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.312808259Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-05T06:28:33.357 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.31333124Z level=info msg="Executing migration" id="create user role table" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.314232122Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.315064212Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.315931315Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.31635905Z level=info msg="Executing migration" id="create builtin role table" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.317223314Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.318023332Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.319004421Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.320111353Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.320963068Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.32194864Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.322764853Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.323348757Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-05T06:28:33.358 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.324301727Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.325018005Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.325989235Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.326385077Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.327244155Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.328198465Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.328728185Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.329736539Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.330455208Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.331448632Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.331920664Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.332895325Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.333651688Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.334665661Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-05T06:28:33.359 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.334977261Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.335850687Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.336891497Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.337587416Z level=info msg="Executing migration" id="add correlation config column" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.338646702Z level=info msg="Executing migration" id="create entity_events table" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.339552695Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.340397637Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.340501977Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.343049878Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.343147606Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.343423608Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.346108926Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.346747316Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-05T06:28:33.360 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.34873632Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T06:28:33.361 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.351423319Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-05T06:28:33.361 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.351890695Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T06:28:33.704 INFO:teuthology.orchestra.run.smithi122.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-05T06:28:33.704 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-05T06:28:33.704 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme1n1 ssd Linux_12473900da27e5869e4c 89.4G Yes 2m ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme2n1 ssd Linux_e685cf3c172ef9c7b59f 89.4G Yes 2m ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme3n1 ssd Linux_771524f7e64fc963e6a0 89.4G Yes 2m ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme4n1 ssd Linux_a04bfe6ff2b181ad27e8 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 103s ago Has a FileSystem, LVM detected 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 /dev/nvme1n1 ssd Linux_78345f66dd76eaa31772 89.4G Yes 103s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 /dev/nvme2n1 ssd Linux_312a3bc237f8cd5e1de4 89.4G Yes 103s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 /dev/nvme3n1 ssd Linux_93639b3d818b083c4439 89.4G Yes 103s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi142 /dev/nvme4n1 ssd Linux_d56b3f5a1e7ac519be6a 89.4G No 103s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 78s ago Has a FileSystem, LVM detected 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 /dev/nvme1n1 ssd Linux_a2bb6e9c603247bfe352 89.4G Yes 78s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 /dev/nvme2n1 ssd Linux_117b8ef191368d3b1105 89.4G Yes 78s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 /dev/nvme3n1 ssd Linux_b973b9cc1e929148e77c 89.4G Yes 78s ago 2024-02-05T06:28:33.705 INFO:teuthology.orchestra.run.smithi122.stdout:smithi196 /dev/nvme4n1 ssd Linux_924adb21b2d0d54656e8 89.4G No 78s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.354474848Z level=info msg="Executing migration" id="Drop public config table" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.355328956Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.35614249Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.356668223Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.357528018Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.358358896Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.362142937Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.363144595Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.364153447Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.36510935Z level=info msg="Executing migration" id="add share column" 2024-02-05T06:28:33.741 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.366154076Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.366610213Z level=info msg="Executing migration" id="create file table" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.367616195Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.368308487Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.36877265Z level=info msg="Executing migration" id="create file_meta table" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.369215739Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.370022057Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.370843051Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.371825846Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.372323607Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.373186503Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.374226239Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.374544788Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.375458174Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-05T06:28:33.742 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.375855055Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.37670278Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.377659574Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.378792274Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.379823039Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.380775744Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.390715507Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.39155515Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.392030616Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.3955784Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.396515417Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.397269825Z level=info msg="Executing migration" id="create folder table" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.397700659Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.400587234Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.403407492Z level=info msg="Executing migration" id="Update folder title length" 2024-02-05T06:28:33.743 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.406299513Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=migrator t=2024-02-05T06:28:33.406751701Z level=info msg="migrations completed" performed=485 skipped=0 duration=647.917005ms 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore t=2024-02-05T06:28:33.407155758Z level=info msg="Created default organization" 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=plugin.loader t=2024-02-05T06:28:33.46446171Z level=info msg="Plugin registered" pluginID=input 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=plugin.loader t=2024-02-05T06:28:33.491600952Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=plugin.loader t=2024-02-05T06:28:33.491620978Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=secrets t=2024-02-05T06:28:33.491653209Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=query_data t=2024-02-05T06:28:33.493759404Z level=info msg="Query Service initialization" 2024-02-05T06:28:33.744 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=live.push_http t=2024-02-05T06:28:33.496552933Z level=info msg="Live Push Gateway initialization" 2024-02-05T06:28:34.016 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:33 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.016 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:33 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.016 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:33 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.016 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:33 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.017 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:33 smithi122 ceph-mon[25512]: Deploying daemon node-exporter.a on smithi122 2024-02-05T06:28:34.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:33 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:33 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:33 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:33 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:33 smithi196 ceph-mon[30516]: Deploying daemon node-exporter.a on smithi122 2024-02-05T06:28:34.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:34.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:33 smithi142 ceph-mon[30053]: Deploying daemon node-exporter.a on smithi122 2024-02-05T06:28:34.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-05T06:28:34.562 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=infra.usagestats.collector t=2024-02-05T06:28:34.10169338Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-05T06:28:34.562 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=provisioning.datasources t=2024-02-05T06:28:34.102653061Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=provisioning.datasources t=2024-02-05T06:28:34.113668499Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=provisioning.alerting t=2024-02-05T06:28:34.124504283Z level=info msg="starting to provision alerting" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=provisioning.alerting t=2024-02-05T06:28:34.124527314Z level=info msg="finished to provision alerting" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=grafanaStorageLogger t=2024-02-05T06:28:34.124697596Z level=info msg="storage starting" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=ngalert.state.manager t=2024-02-05T06:28:34.124881257Z level=info msg="Warming state cache for startup" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=http.server t=2024-02-05T06:28:34.126503355Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=ngalert.state.manager t=2024-02-05T06:28:34.132047143Z level=info msg="State cache has been initialized" states=0 duration=7.166205ms 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=ticker t=2024-02-05T06:28:34.132097846Z level=info msg=starting first_tick=2024-02-05T06:28:40Z 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=ngalert.multiorg.alertmanager t=2024-02-05T06:28:34.132109834Z level=info msg="starting MultiOrg Alertmanager" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore.transactions t=2024-02-05T06:28:34.142744406Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore.transactions t=2024-02-05T06:28:34.143808764Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore.transactions t=2024-02-05T06:28:34.152958871Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-02-05T06:28:34.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a[42314]: logger=sqlstore.transactions t=2024-02-05T06:28:34.163657319Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-02-05T06:28:34.721 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:34.795 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:34 smithi122 bash[56290]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T06:28:35.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-mon[30053]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:35.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:34 smithi142 ceph-mon[30053]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:35.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:34 smithi122 ceph-mon[25512]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:35.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:34 smithi122 ceph-mon[25512]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:35.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:34 smithi196 ceph-mon[30516]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:35.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:34 smithi196 ceph-mon[30516]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:35.581 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:35 smithi122 bash[56290]: Getting image source signatures 2024-02-05T06:28:35.581 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:35 smithi122 bash[56290]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T06:28:35.581 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:35 smithi122 bash[56290]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T06:28:35.581 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:35 smithi122 bash[56290]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T06:28:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:35 smithi196 ceph-mon[30516]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:35 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:36.284 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:35 smithi122 ceph-mon[25512]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:36.284 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:35 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:36.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:35 smithi142 ceph-mon[30053]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:36.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:35 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: alertmanager 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: alertmanager 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: count: 1 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi122=a 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:27:29.947671Z' 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: ports: 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: - 9093 2024-02-05T06:28:37.239 INFO:teuthology.orchestra.run.smithi122.stdout: - 9094 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: running: 0 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: size: 1 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:events: 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:- 2024-02-05T06:28:18.057085Z service:alertmanager [INFO] "service was created" 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: grafana 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: grafana 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: count: 1 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi142=a 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout:spec: 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: anonymous_access: true 2024-02-05T06:28:37.240 INFO:teuthology.orchestra.run.smithi122.stdout: protocol: https 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:27:32.777914Z' 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: ports: 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: - 3000 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: running: 0 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: size: 1 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:events: 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:- 2024-02-05T06:28:32.795224Z service:grafana [INFO] "service was created" 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: mgr 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: mgr 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: count: 3 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.241 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi122=y 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi142=x 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi196=z 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:25:56.173328Z' 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: last_refresh: '2024-02-05T06:27:56.598163Z' 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: running: 3 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: size: 3 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: mon 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: mon 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: count: 3 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi122:172.21.15.122=a 2024-02-05T06:28:37.242 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi142:172.21.15.142=b 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi196:172.21.15.196=c 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:25:28.662348Z' 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: last_refresh: '2024-02-05T06:27:56.598051Z' 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: running: 3 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: size: 3 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: node-exporter 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: node-exporter 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: count: 3 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi122=a 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi142=b 2024-02-05T06:28:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi196=c 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:27:27.106102Z' 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: ports: 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: - 9100 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: running: 0 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: size: 3 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: osd 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: osd 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:unmanaged: true 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:spec: 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: filter_logic: AND 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout: objectstore: bluestore 2024-02-05T06:28:37.244 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: container_image_id: 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: last_refresh: '2024-02-05T06:27:56.598247Z' 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: running: 3 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: size: 3 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: prometheus 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: prometheus 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: count: 1 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi142=a 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:27:24.379091Z' 2024-02-05T06:28:37.245 INFO:teuthology.orchestra.run.smithi122.stdout: last_refresh: '2024-02-05T06:27:57.605314Z' 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: ports: 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: - 9095 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: running: 1 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: size: 1 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:--- 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:service_type: rgw 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:service_id: foo.a 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:service_name: rgw.foo.a 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:placement: 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: count: 1 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: hosts: 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: - smithi122=foo.a 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout:status: 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: created: '2024-02-05T06:27:25.231618Z' 2024-02-05T06:28:37.246 INFO:teuthology.orchestra.run.smithi122.stdout: last_refresh: '2024-02-05T06:28:03.856973Z' 2024-02-05T06:28:37.247 INFO:teuthology.orchestra.run.smithi122.stdout: ports: 2024-02-05T06:28:37.247 INFO:teuthology.orchestra.run.smithi122.stdout: - 80 2024-02-05T06:28:37.247 INFO:teuthology.orchestra.run.smithi122.stdout: running: 1 2024-02-05T06:28:37.247 INFO:teuthology.orchestra.run.smithi122.stdout: size: 1 2024-02-05T06:28:37.250 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:37 smithi122 bash[56290]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T06:28:37.581 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:37 smithi122 bash[56290]: Writing manifest to image destination 2024-02-05T06:28:38.081 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:37 smithi122 podman[56290]: 2024-02-05 06:28:37.772476975 +0000 UTC m=+3.216140219 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T06:28:38.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:37 smithi122 ceph-mon[25512]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:38.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:37 smithi122 ceph-mon[25512]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-05T06:28:38.249 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-05T06:28:38.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:37 smithi196 ceph-mon[30516]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:38.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:37 smithi196 ceph-mon[30516]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-05T06:28:38.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:37 smithi142 ceph-mon[30053]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:38.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:37 smithi142 ceph-mon[30053]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-05T06:28:38.536 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 podman[56290]: 2024-02-05 06:28:38.30515137 +0000 UTC m=+3.748814669 container create f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:38.536 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 podman[56290]: 2024-02-05 06:28:38.540773163 +0000 UTC m=+3.984436414 container init f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:38.727 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T06:28:38.831 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 podman[56290]: 2024-02-05 06:28:38.546262599 +0000 UTC m=+3.989925840 container start f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.549Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.550Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.550Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.550Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.550Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T06:28:38.832 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T06:28:38.833 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T06:28:38.834 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T06:28:38.835 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a[56554]: ts=2024-02-05T06:28:38.551Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 bash[56290]: f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 2024-02-05T06:28:38.836 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:28:38 smithi122 systemd[1]: Started Ceph node-exporter.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:39.534 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:39 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: ::ffff:172.21.15.142 - - [05/Feb/2024:06:28:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T06:28:39.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:39 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:39.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:39 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:39.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:39 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:39.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:39 smithi122 ceph-mon[25512]: Deploying daemon node-exporter.b on smithi142 2024-02-05T06:28:40.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:39 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:39 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:39 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:39 smithi196 ceph-mon[30516]: Deploying daemon node-exporter.b on smithi142 2024-02-05T06:28:40.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:39 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:39 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:39 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' 2024-02-05T06:28:40.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:39 smithi142 ceph-mon[30053]: Deploying daemon node-exporter.b on smithi142 2024-02-05T06:28:40.063 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:39 smithi142 bash[42594]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T06:28:40.722 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:40 smithi142 bash[42594]: Getting image source signatures 2024-02-05T06:28:40.722 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:40 smithi142 bash[42594]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T06:28:40.722 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:40 smithi142 bash[42594]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T06:28:40.722 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:40 smithi142 bash[42594]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T06:28:40.782 INFO:teuthology.orchestra.run.smithi122.stdout:osd 3 44s ago - 2024-02-05T06:28:41.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:40 smithi196 ceph-mon[30516]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:41.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:40 smithi196 ceph-mon[30516]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:41.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:40 smithi122 ceph-mon[25512]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:41.061 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:40 smithi122 ceph-mon[25512]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:41.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:40 smithi142 ceph-mon[30053]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-05T06:28:41.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:40 smithi142 ceph-mon[30053]: from='mgr.14421 172.21.15.122:0/4166371311' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T06:28:41.329 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-05T06:28:41.338 INFO:tasks.cephadm:Teardown begin 2024-02-05T06:28:41.338 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:41.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:41.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:28:41.435 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-05T06:28:41.435 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T06:28:41.449 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T06:28:41.464 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T06:28:41.491 INFO:tasks.cephadm:Stopping all daemons... 2024-02-05T06:28:41.491 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-05T06:28:41.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a 2024-02-05T06:28:41.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:41 smithi122 systemd[1]: Stopping Ceph mon.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:41.679 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:41 smithi142 bash[42594]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T06:28:41.679 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:41 smithi142 bash[42594]: Writing manifest to image destination 2024-02-05T06:28:41.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:41 smithi142 ceph-mon[30053]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:42.000 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:41 smithi142 podman[42594]: 2024-02-05 06:28:41.889102298 +0000 UTC m=+2.122630981 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T06:28:42.000 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 podman[42594]: 2024-02-05 06:28:42.004336598 +0000 UTC m=+2.237865282 container create 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:28:42.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:41 smithi196 ceph-mon[30516]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:42.081 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE Bus STOPPING 2024-02-05T06:28:42.081 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T06:28:42.081 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE Bus STOPPED 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE Bus STARTING 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE Serving on http://:::9283 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:41] ENGINE Bus STARTED 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-mon[25512]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a[25489]: 2024-02-05T06:28:41.805+0000 7f43b0367700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:41 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a[25489]: 2024-02-05T06:28:41.805+0000 7f43b0367700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-05T06:28:42.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:41 smithi122 podman[56852]: 2024-02-05 06:28:41.866952227 +0000 UTC m=+0.184618739 container died b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 podman[42594]: 2024-02-05 06:28:42.239672249 +0000 UTC m=+2.473200934 container init 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 podman[42594]: 2024-02-05 06:28:42.244790938 +0000 UTC m=+2.478319614 container start 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.248Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.248Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.249Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.249Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.249Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T06:28:42.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T06:28:42.314 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T06:28:42.315 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T06:28:42.316 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T06:28:42.317 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b[42679]: ts=2024-02-05T06:28:42.250Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T06:28:42.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:42 smithi122 podman[56852]: 2024-02-05 06:28:42.244453916 +0000 UTC m=+0.562120428 container cleanup b1b1301aec2516ada5f0cbb804927307c52fb28e21c8b7c95e5bd4d2de56d0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T06:28:42.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Feb 05 06:28:42 smithi122 bash[56852]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-a 2024-02-05T06:28:42.633 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.a.service' 2024-02-05T06:28:42.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:42.683 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-05T06:28:42.683 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-05T06:28:42.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.b 2024-02-05T06:28:42.708 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 bash[42594]: 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 2024-02-05T06:28:42.708 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:28:42 smithi142 systemd[1]: Started Ceph node-exporter.b for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:42.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:42 smithi142 systemd[1]: Stopping Ceph mon.b for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:43.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b[30030]: 2024-02-05T06:28:42.978+0000 7f8e4185f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:28:43.312 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:42 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b[30030]: 2024-02-05T06:28:42.978+0000 7f8e4185f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-05T06:28:43.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 podman[42757]: 2024-02-05 06:28:43.132175827 +0000 UTC m=+0.252629770 container died d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-05T06:28:43.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:42 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:42] ENGINE Bus STOPPING 2024-02-05T06:28:43.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T06:28:43.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:43] ENGINE Bus STOPPED 2024-02-05T06:28:43.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:43] ENGINE Bus STARTING 2024-02-05T06:28:43.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:43] ENGINE Serving on http://:::9283 2024-02-05T06:28:43.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:43 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: [05/Feb/2024:06:28:43] ENGINE Bus STARTED 2024-02-05T06:28:43.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 podman[42757]: 2024-02-05 06:28:43.47561853 +0000 UTC m=+0.596072473 container cleanup d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-05T06:28:43.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 bash[42757]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b 2024-02-05T06:28:43.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.b.service' 2024-02-05T06:28:43.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 podman[42775]: 2024-02-05 06:28:43.659938128 +0000 UTC m=+0.524118298 container remove d2eed94ac3c19944d7413defee58edf8afe6dd3ecb60cdd141cc1ac84a878adf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131) 2024-02-05T06:28:43.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-05T06:28:43.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 systemd[1]: Stopped Ceph mon.b for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:43.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Feb 05 06:28:43 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.b.service: Consumed 3.447s CPU time. 2024-02-05T06:28:43.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:43.946 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-05T06:28:43.946 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-05T06:28:43.946 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.c 2024-02-05T06:28:44.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 systemd[1]: Stopping Ceph mon.c for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:44.764 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c[30484]: 2024-02-05T06:28:44.328+0000 7f562c9f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:28:44.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c[30484]: 2024-02-05T06:28:44.328+0000 7f562c9f6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-05T06:28:44.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 podman[43525]: 2024-02-05 06:28:44.482900341 +0000 UTC m=+0.331810390 container died cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-05T06:28:45.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 podman[43525]: 2024-02-05 06:28:44.968170997 +0000 UTC m=+0.817081088 container cleanup cbd9c00638612ab096e1278d58ea3d078e3b54604ce93bb435a12533a7aa4480 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS) 2024-02-05T06:28:45.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Feb 05 06:28:44 smithi196 bash[43525]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mon-c 2024-02-05T06:28:45.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mon.c.service' 2024-02-05T06:28:45.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:45.394 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-05T06:28:45.394 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-05T06:28:45.395 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y 2024-02-05T06:28:45.708 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:45 smithi122 systemd[1]: Stopping Ceph mgr.y for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:45.964 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:45 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:28:45.710+0000 7f25cdaa1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:28:45.964 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:45 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y[25767]: 2024-02-05T06:28:45.710+0000 7f25cdaa1700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-05T06:28:45.964 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:45 smithi122 podman[57003]: 2024-02-05 06:28:45.744269147 +0000 UTC m=+0.148224581 container died 9bd3bd6892db766e38730976234dc2bdba74ea03e9e4c0372147981724d9e5a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:28:46.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:46 smithi122 podman[57003]: 2024-02-05 06:28:46.092813817 +0000 UTC m=+0.496769250 container cleanup 9bd3bd6892db766e38730976234dc2bdba74ea03e9e4c0372147981724d9e5a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T06:28:46.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Feb 05 06:28:46 smithi122 bash[57003]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-y 2024-02-05T06:28:46.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-05T06:28:46.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:46.545 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-05T06:28:46.545 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-05T06:28:46.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x 2024-02-05T06:28:47.062 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:46 smithi142 systemd[1]: Stopping Ceph mgr.x for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:47.063 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:28:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:28:46.662Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.122:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.122:8765: connect: connection refused" 2024-02-05T06:28:47.063 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:28:46 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:28:46.662Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.122:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.122:8765: connect: connection refused" 2024-02-05T06:28:48.562 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:48 smithi142 podman[42890]: 2024-02-05 06:28:48.299035011 +0000 UTC m=+1.562055863 container died 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-05T06:28:49.802 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:49 smithi142 podman[42890]: 2024-02-05 06:28:49.605455694 +0000 UTC m=+2.868476588 container cleanup 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T06:28:49.802 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:49 smithi142 bash[42890]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x 2024-02-05T06:28:50.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-05T06:28:50.057 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:49 smithi142 podman[42910]: 2024-02-05 06:28:49.806591401 +0000 UTC m=+1.507994333 container remove 80e1952f7881b401f7bc8bc032843caff12d688266455734d0c62eea90a69040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-x, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-05T06:28:50.058 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:49 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-02-05T06:28:50.058 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:50 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-02-05T06:28:50.058 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:50 smithi142 systemd[1]: Stopped Ceph mgr.x for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:28:50.058 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Feb 05 06:28:50 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.x.service: Consumed 16.439s CPU time. 2024-02-05T06:28:50.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:50.071 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-05T06:28:50.071 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-05T06:28:50.071 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.z 2024-02-05T06:28:50.400 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:28:50 smithi196 systemd[1]: Stopping Ceph mgr.z for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:50.677 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:28:50 smithi196 podman[43656]: 2024-02-05 06:28:50.403826885 +0000 UTC m=+0.140644918 container died 1c0ae5d7332e98e036633225b65aab3759ea4bcb5f005ced764eae7f1b9aa520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-05T06:28:51.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:28:50 smithi196 podman[43656]: 2024-02-05 06:28:50.76341502 +0000 UTC m=+0.500233061 container cleanup 1c0ae5d7332e98e036633225b65aab3759ea4bcb5f005ced764eae7f1b9aa520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-05T06:28:51.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Feb 05 06:28:50 smithi196 bash[43656]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-mgr-z 2024-02-05T06:28:51.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@mgr.z.service' 2024-02-05T06:28:51.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:51.213 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-05T06:28:51.214 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-05T06:28:51.214 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.0 2024-02-05T06:28:51.503 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:51 smithi122 systemd[1]: Stopping Ceph osd.0 for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:28:51.831 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0[40537]: 2024-02-05T06:28:51.507+0000 7f8d915e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:28:51.832 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0[40537]: 2024-02-05T06:28:51.507+0000 7f8d915e3700 -1 osd.0 32 *** Got signal Terminated *** 2024-02-05T06:28:51.832 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:51 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0[40537]: 2024-02-05T06:28:51.507+0000 7f8d915e3700 -1 osd.0 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T06:28:57.303 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:57 smithi122 podman[57135]: 2024-02-05 06:28:57.043691186 +0000 UTC m=+5.631438354 container died f30e829374acb5fcc88303227ce3aa2ce7b96bf01b1de61ec13271c4acfe964b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:28:57.581 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:57 smithi122 podman[57135]: 2024-02-05 06:28:57.463823527 +0000 UTC m=+6.051570695 container cleanup f30e829374acb5fcc88303227ce3aa2ce7b96bf01b1de61ec13271c4acfe964b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T06:28:57.582 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:57 smithi122 bash[57135]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0 2024-02-05T06:28:58.071 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:57 smithi122 podman[57158]: 2024-02-05 06:28:57.673654178 +0000 UTC m=+0.626334421 container remove f30e829374acb5fcc88303227ce3aa2ce7b96bf01b1de61ec13271c4acfe964b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:28:58.332 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:57.978510161 +0000 UTC m=+0.018548695 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:28:58.332 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:58.082571805 +0000 UTC m=+0.122610335 container create 17cfaadb649d7d28d0933ca38765a2e1414989806b72541d4a6391daeea9817d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0-deactivate, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T06:28:58.332 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:58.285421886 +0000 UTC m=+0.325460421 container init 17cfaadb649d7d28d0933ca38765a2e1414989806b72541d4a6391daeea9817d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=) 2024-02-05T06:28:58.332 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:58.291123728 +0000 UTC m=+0.331162260 container start 17cfaadb649d7d28d0933ca38765a2e1414989806b72541d4a6391daeea9817d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T06:28:58.831 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:58.38516096 +0000 UTC m=+0.425199498 container attach 17cfaadb649d7d28d0933ca38765a2e1414989806b72541d4a6391daeea9817d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T06:28:58.832 INFO:journalctl@ceph.osd.0.smithi122.stdout:Feb 05 06:28:58 smithi122 podman[57253]: 2024-02-05 06:28:58.588228984 +0000 UTC m=+0.628267517 container died 17cfaadb649d7d28d0933ca38765a2e1414989806b72541d4a6391daeea9817d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-05T06:28:59.232 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.0.service' 2024-02-05T06:28:59.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:28:59.275 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-05T06:28:59.275 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-05T06:28:59.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.1 2024-02-05T06:28:59.562 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:28:59 smithi142 systemd[1]: Stopping Ceph osd.1 for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:00.063 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:28:59 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1[34609]: 2024-02-05T06:28:59.575+0000 7f20e3635700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:29:00.063 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:28:59 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1[34609]: 2024-02-05T06:28:59.575+0000 7f20e3635700 -1 osd.1 32 *** Got signal Terminated *** 2024-02-05T06:29:00.063 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:28:59 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1[34609]: 2024-02-05T06:28:59.575+0000 7f20e3635700 -1 osd.1 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T06:29:05.231 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:04 smithi142 podman[43026]: 2024-02-05 06:29:04.972878664 +0000 UTC m=+5.490169947 container died aac4633c4da2e112089dedb91ea35ea874c1a72343842ef6a20f322fc84a514f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-05T06:29:05.488 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:05 smithi142 podman[43026]: 2024-02-05 06:29:05.30796562 +0000 UTC m=+5.825256894 container cleanup aac4633c4da2e112089dedb91ea35ea874c1a72343842ef6a20f322fc84a514f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-05T06:29:05.488 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:05 smithi142 bash[43026]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1 2024-02-05T06:29:05.812 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:05 smithi142 podman[43060]: 2024-02-05 06:29:05.492201795 +0000 UTC m=+0.510798174 container remove aac4633c4da2e112089dedb91ea35ea874c1a72343842ef6a20f322fc84a514f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T06:29:06.313 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:05 smithi142 podman[43156]: 2024-02-05 06:29:05.901090608 +0000 UTC m=+0.094913650 container create 7c06ffd7666246f1f159f2ba0e0122bf322f73ac23640199c4b9bb2e344dcd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131) 2024-02-05T06:29:06.313 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:05 smithi142 podman[43156]: 2024-02-05 06:29:05.823784833 +0000 UTC m=+0.017607880 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:29:06.313 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:06 smithi142 podman[43156]: 2024-02-05 06:29:06.061559422 +0000 UTC m=+0.255382465 container init 7c06ffd7666246f1f159f2ba0e0122bf322f73ac23640199c4b9bb2e344dcd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-02-05T06:29:06.313 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:06 smithi142 podman[43156]: 2024-02-05 06:29:06.067084995 +0000 UTC m=+0.260908039 container start 7c06ffd7666246f1f159f2ba0e0122bf322f73ac23640199c4b9bb2e344dcd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-05T06:29:06.313 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:06 smithi142 podman[43156]: 2024-02-05 06:29:06.144764317 +0000 UTC m=+0.338587377 container attach 7c06ffd7666246f1f159f2ba0e0122bf322f73ac23640199c4b9bb2e344dcd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-05T06:29:06.651 INFO:journalctl@ceph.osd.1.smithi142.stdout:Feb 05 06:29:06 smithi142 podman[43156]: 2024-02-05 06:29:06.363248594 +0000 UTC m=+0.557071637 container died 7c06ffd7666246f1f159f2ba0e0122bf322f73ac23640199c4b9bb2e344dcd6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-05T06:29:06.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.1.service' 2024-02-05T06:29:06.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:29:06.971 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-05T06:29:06.972 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-05T06:29:06.972 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.2 2024-02-05T06:29:07.272 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:07 smithi196 systemd[1]: Stopping Ceph osd.2 for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:07.772 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:07 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2[35820]: 2024-02-05T06:29:07.275+0000 7f66b0fd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:29:07.773 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:07 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2[35820]: 2024-02-05T06:29:07.275+0000 7f66b0fd0700 -1 osd.2 32 *** Got signal Terminated *** 2024-02-05T06:29:07.773 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:07 smithi196 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2[35820]: 2024-02-05T06:29:07.275+0000 7f66b0fd0700 -1 osd.2 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T06:29:13.023 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:12 smithi196 podman[43789]: 2024-02-05 06:29:12.757174827 +0000 UTC m=+5.582915200 container died 2b895baf26273e9af6851aa72ee9baad6b64769f270bb63df70e8cb97d95875e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-05T06:29:13.403 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43789]: 2024-02-05 06:29:13.114093999 +0000 UTC m=+5.939834365 container cleanup 2b895baf26273e9af6851aa72ee9baad6b64769f270bb63df70e8cb97d95875e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-05T06:29:13.403 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 bash[43789]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2 2024-02-05T06:29:13.721 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43821]: 2024-02-05 06:29:13.406806605 +0000 UTC m=+0.645523270 container remove 2b895baf26273e9af6851aa72ee9baad6b64769f270bb63df70e8cb97d95875e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T06:29:14.023 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43921]: 2024-02-05 06:29:13.627658946 +0000 UTC m=+0.018363096 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T06:29:14.023 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43921]: 2024-02-05 06:29:13.743910396 +0000 UTC m=+0.134614546 container create 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-05T06:29:14.023 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43921]: 2024-02-05 06:29:13.960338704 +0000 UTC m=+0.351042850 container init 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-05T06:29:14.023 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:13 smithi196 podman[43921]: 2024-02-05 06:29:13.965891577 +0000 UTC m=+0.356595728 container start 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T06:29:14.523 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:14 smithi196 podman[43921]: 2024-02-05 06:29:14.043309994 +0000 UTC m=+0.434014150 container attach 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-05T06:29:14.523 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:14 smithi196 podman[43921]: 2024-02-05 06:29:14.273222703 +0000 UTC m=+0.663926848 container died 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-05T06:29:14.808 INFO:journalctl@ceph.osd.2.smithi196.stdout:Feb 05 06:29:14 smithi196 podman[43967]: 2024-02-05 06:29:14.796050367 +0000 UTC m=+0.517878558 container remove 728e5160646d63b22e06bc94efd0eae9010e39a1e1ff8605010f3c6a6ea1ae75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-05T06:29:14.813 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@osd.2.service' 2024-02-05T06:29:14.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:29:14.852 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-05T06:29:14.852 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-05T06:29:14.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-05T06:29:15.132 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:14 smithi122 systemd[1]: Stopping Ceph rgw.foo.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:15.582 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a[47344]: 2024-02-05T06:29:15.134+0000 7f0276942700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T06:29:15.582 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:15 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a[47344]: 2024-02-05T06:29:15.134+0000 7f02851adac0 -1 shutting down 2024-02-05T06:29:25.413 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 bash[57397]: time="2024-02-05T06:29:25Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-05T06:29:25.414 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 podman[57397]: 2024-02-05 06:29:25.155465431 +0000 UTC m=+10.100812179 container died 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-05T06:29:25.779 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 podman[57397]: 2024-02-05 06:29:25.590324643 +0000 UTC m=+10.535671435 container cleanup 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True) 2024-02-05T06:29:25.780 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 bash[57397]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-05T06:29:26.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-05T06:29:26.036 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 podman[57422]: 2024-02-05 06:29:25.783322901 +0000 UTC m=+0.623044114 container remove 99f77f441ffe124e623fdaf89f2c5f87a5310ae7b5dc3464943565504ad1e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-05T06:29:26.036 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:25 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-05T06:29:26.036 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:26 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2024-02-05T06:29:26.036 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:26 smithi122 systemd[1]: Stopped Ceph rgw.foo.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:29:26.036 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Feb 05 06:29:26 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@rgw.foo.a.service: Consumed 1.377s CPU time. 2024-02-05T06:29:26.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:29:26.049 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-05T06:29:26.049 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-05T06:29:26.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@prometheus.a 2024-02-05T06:29:26.339 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 systemd[1]: Stopping Ceph prometheus.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:26.595 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.342Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-05T06:29:26.595 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.342Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.342Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a[40437]: ts=2024-02-05T06:29:26.343Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-05T06:29:26.596 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 podman[43287]: 2024-02-05 06:29:26.353100726 +0000 UTC m=+0.109654926 container died 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:26.944 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 podman[43287]: 2024-02-05 06:29:26.697698757 +0000 UTC m=+0.454252999 container cleanup 2e3df5e22d483d9146b17c87a037bce4b23c9d492f8ced6cdd2054b6c07637c0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:26.945 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Feb 05 06:29:26 smithi142 bash[43287]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-prometheus-a 2024-02-05T06:29:27.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-05T06:29:27.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T06:29:27.160 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-05T06:29:27.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T06:29:27.347 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:28.332 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:28 smithi122 systemd[1]: Stopping Ceph alertmanager.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:28.702 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:28 smithi122 ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a[54751]: ts=2024-02-05T06:29:28.431Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-05T06:29:28.702 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:28 smithi122 podman[57742]: 2024-02-05 06:29:28.452278067 +0000 UTC m=+0.123600411 container died 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:29.051 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:28 smithi122 podman[57742]: 2024-02-05 06:29:28.870921886 +0000 UTC m=+0.542244230 container cleanup 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:29.051 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:28 smithi122 bash[57742]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a 2024-02-05T06:29:29.318 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:29 smithi122 podman[57761]: 2024-02-05 06:29:29.055449595 +0000 UTC m=+0.606330895 container remove 3a20399ed4fc887958e17a0365c323ee46c7fb2a0cafdaa34532b926105d1ca5 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:29.318 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:29 smithi122 podman[57761]: 2024-02-05 06:29:29.131039764 +0000 UTC m=+0.681921018 volume remove c683a79c40b253eeb8966c647466365c6278d9b91174c796fb9c8b024a66411f 2024-02-05T06:29:29.755 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:29 smithi122 systemd[1]: Stopping Ceph node-exporter.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:29.756 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:29 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-05T06:29:29.756 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Feb 05 06:29:29 smithi122 systemd[1]: Stopped Ceph alertmanager.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:29:30.081 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:29 smithi122 podman[57880]: 2024-02-05 06:29:29.805220523 +0000 UTC m=+0.113287777 container died f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:30.332 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 podman[57880]: 2024-02-05 06:29:30.183531651 +0000 UTC m=+0.491598913 container cleanup f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:30.332 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 bash[57880]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a 2024-02-05T06:29:30.739 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 podman[57903]: 2024-02-05 06:29:30.443505866 +0000 UTC m=+0.639635670 container remove f5c84b57c14e0796d75fb487bafc19ae2dafd7d8e055da239bb6b7273a4f5d62 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T06:29:30.740 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T06:29:30.740 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-05T06:29:30.740 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 systemd[1]: Stopped Ceph node-exporter.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:29:30.740 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Feb 05 06:29:30 smithi122 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 1.769s CPU time. 2024-02-05T06:29:31.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T06:29:31.415 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:32.563 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:32 smithi142 systemd[1]: Stopping Ceph grafana.a for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:32.564 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:32 smithi142 podman[43636]: 2024-02-05 06:29:32.558200976 +0000 UTC m=+0.095529509 container died 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, distribution-scope=public, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, version=8.5, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, name=ubi8, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-02-05T06:29:33.156 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:32 smithi142 podman[43636]: 2024-02-05 06:29:32.90105558 +0000 UTC m=+0.438384102 container cleanup 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, release=239.1651231664, io.buildah.version=1.32.0, maintainer=Paul Cuzner , distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, vcs-type=git) 2024-02-05T06:29:33.156 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:32 smithi142 bash[43636]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a 2024-02-05T06:29:33.500 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:33 smithi142 podman[43655]: 2024-02-05 06:29:33.160089798 +0000 UTC m=+0.604705376 container remove 55f5ec49b207946926d4e7355bfcf9769b656b7ab2fa35b1ed64627a2dfb91f1 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, release=239.1651231664, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., architecture=x86_64, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-02-05T06:29:33.500 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:33 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T06:29:33.500 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:33 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-05T06:29:33.500 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:33 smithi142 systemd[1]: Stopped Ceph grafana.a for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:29:33.500 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Feb 05 06:29:33 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@grafana.a.service: Consumed 2.687s CPU time. 2024-02-05T06:29:33.756 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:33 smithi142 systemd[1]: Stopping Ceph node-exporter.b for f68cd702-c3ee-11ee-95b6-87774f69a715... 2024-02-05T06:29:34.011 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:33 smithi142 podman[43774]: 2024-02-05 06:29:33.759980391 +0000 UTC m=+0.121130857 container died 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:29:34.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 podman[43774]: 2024-02-05 06:29:34.113138937 +0000 UTC m=+0.474289405 container cleanup 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:29:34.313 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 bash[43774]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b 2024-02-05T06:29:34.664 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 podman[43793]: 2024-02-05 06:29:34.372511799 +0000 UTC m=+0.617632815 container remove 9c652e922776d732f415373b7dc2dd28e405b4e001a9a9c182bb52145cdefdb3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f68cd702-c3ee-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T06:29:34.664 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-05T06:29:34.664 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-05T06:29:34.664 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 systemd[1]: Stopped Ceph node-exporter.b for f68cd702-c3ee-11ee-95b6-87774f69a715. 2024-02-05T06:29:34.664 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Feb 05 06:29:34 smithi142 systemd[1]: ceph-f68cd702-c3ee-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 1.723s CPU time. 2024-02-05T06:29:35.136 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T06:29:35.325 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:36.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:29:36.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:29:36.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T06:29:36.519 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-05T06:29:36.521 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi122/crash 2024-02-05T06:29:36.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash -- . 2024-02-05T06:29:36.565 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T06:29:36.565 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-02-05T06:29:36.568 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi142/crash 2024-02-05T06:29:36.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash -- . 2024-02-05T06:29:36.613 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T06:29:36.613 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-02-05T06:29:36.615 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi196/crash 2024-02-05T06:29:36.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash -- . 2024-02-05T06:29:36.660 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T06:29:36.660 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-02-05T06:29:36.661 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-05T06:29:36.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-05T06:29:36.694 INFO:teuthology.orchestra.run.smithi122.stdout:2024-02-05T06:25:47.316850+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-05T06:29:36.694 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-05T06:29:36.694 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[SEC\]' /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-05T06:29:36.764 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]' /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-05T06:29:36.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[WRN\]' /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-05T06:29:36.904 INFO:teuthology.orchestra.run.smithi122.stdout:2024-02-05T06:25:47.316850+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-05T06:29:36.904 INFO:tasks.cephadm:Compressing logs... 2024-02-05T06:29:36.905 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T06:29:36.946 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T06:29:36.949 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T06:29:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T06:29:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-02-05T06:29:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.a.log: /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.y.log: 57.9% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-05T06:29:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr: 82.4%/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log: -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T06:29:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log: 84.9% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.z.log 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T06:29:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log 2024-02-05T06:29:36.978 INFO:teuthology.orchestra.run.smithi122.stderr: 90.5% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log: 89.6% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log: 80.6% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-05T06:29:36.981 INFO:teuthology.orchestra.run.smithi196.stderr: 84.4% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T06:29:36.982 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-02-05T06:29:36.982 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-05T06:29:36.982 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T06:29:36.982 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.x.log: 89.5% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log: /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log: 80.6% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr: 84.2% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T06:29:36.983 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log: 81.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T06:29:36.984 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log: /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.c.log: 90.2% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.z.log.gz 2024-02-05T06:29:36.984 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T06:29:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.b.log: 90.2% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-05T06:29:36.985 INFO:teuthology.orchestra.run.smithi142.stderr: 90.5% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T06:29:37.033 INFO:teuthology.orchestra.run.smithi142.stderr: 92.3% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-05T06:29:37.035 INFO:teuthology.orchestra.run.smithi196.stderr: 92.3% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-05T06:29:37.036 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-05T06:29:37.154 INFO:teuthology.orchestra.run.smithi122.stderr: 91.4% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-05T06:29:37.231 INFO:teuthology.orchestra.run.smithi122.stderr: 94.3% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-05T06:29:37.231 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-05T06:29:37.233 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-02-05T06:29:37.233 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.274s 2024-02-05T06:29:37.233 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m0.502s 2024-02-05T06:29:37.233 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.042s 2024-02-05T06:29:37.234 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-02-05T06:29:37.234 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.270s 2024-02-05T06:29:37.234 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m0.318s 2024-02-05T06:29:37.234 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.034s 2024-02-05T06:29:37.275 INFO:teuthology.orchestra.run.smithi142.stderr: 94.3% -- replaced with /var/log/ceph/f68cd702-c3ee-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-05T06:29:37.277 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-02-05T06:29:37.277 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m0.314s 2024-02-05T06:29:37.277 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m0.362s 2024-02-05T06:29:37.277 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.038s 2024-02-05T06:29:37.277 INFO:tasks.cephadm:Archiving logs... 2024-02-05T06:29:37.278 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi122/log 2024-02-05T06:29:37.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T06:29:37.396 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi142/log 2024-02-05T06:29:37.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T06:29:37.484 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi196/log 2024-02-05T06:29:37.484 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T06:29:37.567 INFO:tasks.cephadm:Removing cluster... 2024-02-05T06:29:37.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force 2024-02-05T06:29:37.746 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:37.859 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force 2024-02-05T06:29:38.040 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:38.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68cd702-c3ee-11ee-95b6-87774f69a715 --force 2024-02-05T06:29:38.343 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: f68cd702-c3ee-11ee-95b6-87774f69a715 2024-02-05T06:29:38.465 INFO:tasks.cephadm:Removing cephadm ... 2024-02-05T06:29:38.465 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-05T06:29:38.480 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-05T06:29:38.495 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-05T06:29:38.509 INFO:tasks.cephadm:Teardown complete 2024-02-05T06:29:38.509 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-05T06:29:38.520 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi122:/dev/vg_nvme/lv_1... 2024-02-05T06:29:38.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo nvme disconnect -n lv_1 2024-02-05T06:29:38.666 INFO:teuthology.orchestra.run.smithi122.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-05T06:29:38.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:38.668 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi122:/dev/vg_nvme/lv_2... 2024-02-05T06:29:38.668 DEBUG:teuthology.orchestra.run.smithi122:> sudo nvme disconnect -n lv_2 2024-02-05T06:29:38.803 INFO:teuthology.orchestra.run.smithi122.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-05T06:29:38.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:38.804 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi122:/dev/vg_nvme/lv_3... 2024-02-05T06:29:38.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo nvme disconnect -n lv_3 2024-02-05T06:29:38.951 INFO:teuthology.orchestra.run.smithi122.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-05T06:29:38.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:38.952 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi122:/dev/vg_nvme/lv_4... 2024-02-05T06:29:38.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo nvme disconnect -n lv_4 2024-02-05T06:29:39.092 INFO:teuthology.orchestra.run.smithi122.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-05T06:29:39.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.093 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-02-05T06:29:39.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/scratch_devs 2024-02-05T06:29:39.120 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi142:/dev/vg_nvme/lv_1... 2024-02-05T06:29:39.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo nvme disconnect -n lv_1 2024-02-05T06:29:39.257 INFO:teuthology.orchestra.run.smithi142.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-05T06:29:39.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.258 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi142:/dev/vg_nvme/lv_2... 2024-02-05T06:29:39.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo nvme disconnect -n lv_2 2024-02-05T06:29:39.413 INFO:teuthology.orchestra.run.smithi142.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-05T06:29:39.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.415 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi142:/dev/vg_nvme/lv_3... 2024-02-05T06:29:39.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo nvme disconnect -n lv_3 2024-02-05T06:29:39.567 INFO:teuthology.orchestra.run.smithi142.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-05T06:29:39.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.569 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi142:/dev/vg_nvme/lv_4... 2024-02-05T06:29:39.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo nvme disconnect -n lv_4 2024-02-05T06:29:39.697 INFO:teuthology.orchestra.run.smithi142.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-05T06:29:39.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.698 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-05T06:29:39.698 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/scratch_devs 2024-02-05T06:29:39.725 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-02-05T06:29:39.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_1 2024-02-05T06:29:39.878 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-05T06:29:39.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:39.880 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-02-05T06:29:39.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_2 2024-02-05T06:29:40.015 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-05T06:29:40.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.017 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-02-05T06:29:40.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_3 2024-02-05T06:29:40.152 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-05T06:29:40.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.153 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-02-05T06:29:40.154 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_4 2024-02-05T06:29:40.301 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-05T06:29:40.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.302 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-02-05T06:29:40.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-02-05T06:29:40.329 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-05T06:29:40.338 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-05T06:29:40.339 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:29:40.341 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:29:40.344 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T06:29:40.355 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:29:40.356 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +721us[ +748us] +/- 91ms 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -1840us[-1813us] +/- 76ms 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +1156us[+1156us] +/- 68ms 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:29:40.361 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-02-05T06:29:40.362 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +712us[ +712us] +/- 91ms 2024-02-05T06:29:40.362 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -1851us[-1850us] +/- 76ms 2024-02-05T06:29:40.362 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +1171us[+1171us] +/- 68ms 2024-02-05T06:29:40.362 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:29:40.383 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-02-05T06:29:40.386 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T06:29:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-02-05T06:29:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +708us[ +708us] +/- 91ms 2024-02-05T06:29:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -1843us[-1833us] +/- 76ms 2024-02-05T06:29:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +1179us[+1179us] +/- 68ms 2024-02-05T06:29:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T06:29:40.388 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-05T06:29:40.404 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-05T06:29:40.405 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-05T06:29:40.436 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T06:29:40.517 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T06:29:40.586 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T06:29:40.654 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:29:40.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:29:40.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.719 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T06:29:40.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:40.749 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-05T06:29:40.759 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-05T06:29:40.771 INFO:teuthology.task.internal:Duration was 676.431345 seconds 2024-02-05T06:29:40.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-05T06:29:40.780 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-05T06:29:40.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T06:29:40.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T06:29:40.786 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T06:29:40.827 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:29:40.831 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:29:40.831 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T06:29:41.123 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-05T06:29:41.123 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-02-05T06:29:41.124 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T06:29:41.145 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-02-05T06:29:41.145 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T06:29:41.172 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-02-05T06:29:41.172 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T06:29:41.199 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-05T06:29:41.199 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T06:29:41.202 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T06:29:41.214 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T06:29:41.267 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-05T06:29:41.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T06:29:41.273 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T06:29:41.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T06:29:41.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-05T06:29:41.471 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-05T06:29:41.472 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T06:29:41.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T06:29:41.526 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T06:29:41.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-05T06:29:41.564 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.589 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-02-05T06:29:41.592 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-02-05T06:29:41.619 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-02-05T06:29:41.643 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:41.675 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:41.689 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T06:29:41.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T06:29:41.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-05T06:29:41.713 INFO:teuthology.task.internal:Transferring archived files... 2024-02-05T06:29:41.714 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi122 2024-02-05T06:29:41.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T06:29:41.753 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi142 2024-02-05T06:29:41.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T06:29:41.792 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689/remote/smithi196 2024-02-05T06:29:41.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T06:29:41.828 INFO:teuthology.task.internal:Removing archive directory... 2024-02-05T06:29:41.828 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T06:29:41.831 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T06:29:41.833 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T06:29:41.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-05T06:29:41.890 INFO:teuthology.task.internal:Not uploading archives. 2024-02-05T06:29:41.890 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-05T06:29:41.921 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-05T06:29:41.921 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T06:29:41.924 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T06:29:41.926 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T06:29:41.935 INFO:teuthology.orchestra.run.smithi122.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 06:29 /home/ubuntu/cephtest 2024-02-05T06:29:41.939 INFO:teuthology.orchestra.run.smithi142.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 06:29 /home/ubuntu/cephtest 2024-02-05T06:29:41.940 INFO:teuthology.orchestra.run.smithi196.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 06:29 /home/ubuntu/cephtest 2024-02-05T06:29:41.941 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-05T06:29:41.996 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-05T06:29:42.021 INFO:teuthology.nuke:Checking targets against current locks 2024-02-05T06:29:42.050 DEBUG:teuthology.nuke:shortname: smithi122 2024-02-05T06:29:42.050 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T06:29:42.076 DEBUG:teuthology.nuke:shortname: smithi142 2024-02-05T06:29:42.077 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T06:29:42.081 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.758647', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMgdWwhNXncLNrk8F7H2F7c+6jNJEs7cdStV5ch07938MyRUC+au+l4ikI9mmlLmK97hL546eK3EzJjTX67qcTU='} 2024-02-05T06:29:42.100 DEBUG:teuthology.nuke:shortname: smithi196 2024-02-05T06:29:42.101 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T06:29:42.106 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.762014', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKtEh0Ft4nT4zvD/RsICi5M88SxKUHRSz7dGoB2If7JcoHK6aacCPMeP0fLdm0KPBpG97CPdNILEFv9IqdblsyA='} 2024-02-05T06:29:42.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 06:09:45.760627', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP3CGnu0UDr1c/RqYaNfvC2ZQaUGg4LVpYRmFOPD5Fr8Ia3YA7pY3Mt6xpEQX1kCKvaU+2BNnjUQK11J4dgcyLI='} 2024-02-05T06:29:42.133 INFO:teuthology.orchestra.console.smithi122:Power off 2024-02-05T06:29:42.134 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T06:29:42.160 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-02-05T06:29:42.160 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:42.177 INFO:teuthology.orchestra.console.smithi142:Power off 2024-02-05T06:29:42.178 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T06:29:42.186 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-02-05T06:29:42.205 INFO:teuthology.orchestra.console.smithi196:Power off 2024-02-05T06:29:42.206 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T06:29:42.215 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-02-05T06:29:42.215 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:42.227 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-02-05T06:29:42.228 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:42.237 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-05T06:29:42.251 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-05T06:29:46.188 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:46.239 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:46.252 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:46.299 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-02-05T06:29:46.349 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-05T06:29:46.361 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-05T06:29:50.304 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:50.350 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:50.362 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:50.414 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-02-05T06:29:50.459 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-05T06:29:50.472 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-02-05T06:29:50.515 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-02-05T06:29:54.462 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:54.473 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T06:29:54.573 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-02-05T06:29:54.583 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-02-05T06:29:54.674 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-02-05T06:29:54.683 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-02-05T06:29:54.787 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 676.4313445091248 failure_reason: '"2024-02-05T06:25:47.316850+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_gabrioux@teuthology success: false 2024-02-05T06:29:54.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T06:29:54.866 INFO:teuthology.run:FAIL