2024-02-10T06:43:41.303 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T06:43:41.303 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} --verbose --archive /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166 --name yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/orig.config.yaml 2024-02-10T06:43:41.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T06:43:41.404 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166 branch: main description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: yweinste@redhat.com first_in_suite: false job_id: '7555166' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1610 sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKb1H6XlSqlcBBqJgKD9uy3eDN5+yr0+g26Xdf2Q0GFF0ttGanEfF3wuF59w++cjDVofVxg5yp1QMG5NXCPCiog= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAh9RQHm7p8CKgkz/ikf6UdICueZGSo11pgyncO2VRdjXA7xaBoSYqQvpxUBsuU79C8IM9bmKCCLTGAFvj0YHs= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGqbhrGVNQZB/4hdAQAJu6Fc2hWkbfn+jUl3uQWkhWV/ClgsrebZWD+Bnb8WDR5sFPziPWMGABPVqlsn8NU1jw= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-09_19:44:16 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-10T06:43:41.404 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa; will attempt to use it 2024-02-10T06:43:41.405 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa/tasks 2024-02-10T06:43:41.405 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-10T06:43:41.407 INFO:teuthology.task.internal:Checking packages... 2024-02-10T06:43:41.432 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90' 2024-02-10T06:43:41.432 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-10T06:43:41.432 INFO:teuthology.packaging:ref: None 2024-02-10T06:43:41.432 INFO:teuthology.packaging:tag: None 2024-02-10T06:43:41.432 INFO:teuthology.packaging:branch: main 2024-02-10T06:43:41.432 INFO:teuthology.packaging:sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:43:41.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-10T06:43:41.683 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1361.gf8e4e331 2024-02-10T06:43:41.685 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-10T06:43:41.691 INFO:teuthology.task.internal:no buildpackages task found 2024-02-10T06:43:41.691 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-10T06:43:41.707 INFO:teuthology.task.internal:Saving configuration 2024-02-10T06:43:41.717 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-10T06:43:41.723 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T06:43:41.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.528159', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKb1H6XlSqlcBBqJgKD9uy3eDN5+yr0+g26Xdf2Q0GFF0ttGanEfF3wuF59w++cjDVofVxg5yp1QMG5NXCPCiog='} 2024-02-10T06:43:41.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.525541', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAh9RQHm7p8CKgkz/ikf6UdICueZGSo11pgyncO2VRdjXA7xaBoSYqQvpxUBsuU79C8IM9bmKCCLTGAFvj0YHs='} 2024-02-10T06:43:41.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.526963', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGqbhrGVNQZB/4hdAQAJu6Fc2hWkbfn+jUl3uQWkhWV/ClgsrebZWD+Bnb8WDR5sFPziPWMGABPVqlsn8NU1jw='} 2024-02-10T06:43:41.785 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-10T06:43:41.791 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-10T06:43:41.791 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-10T06:43:41.791 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-10T06:43:41.791 INFO:teuthology.run_tasks:Running task console_log... 2024-02-10T06:43:41.888 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0bfb99a160>, signals=[15]) 2024-02-10T06:43:41.888 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-10T06:43:41.894 INFO:teuthology.task.internal:Opening connections... 2024-02-10T06:43:41.894 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-02-10T06:43:41.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:43:41.992 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-02-10T06:43:41.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:43:42.077 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-02-10T06:43:42.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:43:42.162 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-10T06:43:42.170 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-02-10T06:43:42.187 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-02-10T06:43:42.187 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-02-10T06:43:42.244 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-02-10T06:43:42.244 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T06:43:42.245 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T06:43:42.246 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-02-10T06:43:42.268 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-02-10T06:43:42.285 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-02-10T06:43:42.285 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-02-10T06:43:42.341 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T06:43:42.342 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T06:43:42.343 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-02-10T06:43:42.367 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-02-10T06:43:42.385 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-02-10T06:43:42.385 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-02-10T06:43:42.441 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T06:43:42.442 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T06:43:42.442 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-02-10T06:43:42.467 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-10T06:43:42.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-10T06:43:42.514 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-10T06:43:42.514 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-02-10T06:43:42.517 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-02-10T06:43:42.519 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-02-10T06:43:42.535 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-10T06:43:42.541 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-10T06:43:42.541 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-02-10T06:43:42.573 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-02-10T06:43:42.576 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-02-10T06:43:42.693 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-10T06:43:42.769 INFO:teuthology.run_tasks:Running task kernel... 2024-02-10T06:43:42.784 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-10T06:43:42.784 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-10T06:43:42.784 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T06:43:42.785 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T06:43:42.785 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T06:43:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:43:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:43:42.802 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-02-10T06:43:42.803 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-02-10T06:43:42.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:43:42.804 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-02-10T06:43:42.859 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-361.el9.x86_64 2024-02-10T06:43:42.859 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.14.0-361.el9.x86_64 2024-02-10T06:43:42.859 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2024-02-10T06:43:42.860 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-361.el9.x86_64 2024-02-10T06:43:42.861 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-361.el9.x86_64 2024-02-10T06:43:42.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2024-02-10T06:43:42.862 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-361.el9.x86_64 2024-02-10T06:43:42.862 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.14.0-361.el9.x86_64 2024-02-10T06:43:42.862 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-02-10T06:43:45.093 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-02-10T06:43:45.123 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-10T06:43:45.156 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-02-10T06:43:46.543 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-02-10T06:43:46.706 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-02-10T06:43:46.765 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-02-10T06:43:47.539 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-02-10T06:43:48.697 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-02-10T06:43:48.826 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-02-10T06:43:51.798 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 19 MB 00:02 2024-02-10T06:43:52.360 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-02-10T06:43:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.9 MB 00:00 2024-02-10T06:43:56.641 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 5.9 MB 00:01 2024-02-10T06:43:57.846 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-02-10T06:43:58.133 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-02-10T06:44:01.216 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 20 MB 00:03 2024-02-10T06:44:04.653 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 20 MB 00:06 2024-02-10T06:44:06.147 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-02-10T06:44:08.429 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T06:44:08.429 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T06:44:08.528 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-02-10T06:44:08.533 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:44:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2024-02-10T06:44:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 99 M 2024-02-10T06:44:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 122 M 2024-02-10T06:44:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-02-10T06:44:09.449 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 6.6 MB/s | 6.0 MB 00:00 2024-02-10T06:44:09.675 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 8.8 kB/s | 1.7 kB 00:00 2024-02-10T06:44:10.299 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 12 MB/s | 21 MB 00:01 2024-02-10T06:44:10.966 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-10T06:44:11.491 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 16 MB/s | 33 MB 00:02 2024-02-10T06:44:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:44:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:Total 34 MB/s | 99 MB 00:02 2024-02-10T06:44:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-02-10T06:44:11.867 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-02-10T06:44:11.868 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-02-10T06:44:11.915 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T06:44:11.915 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T06:44:12.010 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-02-10T06:44:12.019 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-02-10T06:44:12.020 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:44:12.021 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 99 M 2024-02-10T06:44:12.021 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 122 M 2024-02-10T06:44:12.021 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-02-10T06:44:12.351 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-02-10T06:44:12.893 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-02-10T06:44:12.932 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-02-10T06:44:12.933 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-02-10T06:44:13.486 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 30 MB/s | 33 MB 00:01 2024-02-10T06:44:14.035 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 19 MB/s | 39 MB 00:02 2024-02-10T06:44:14.038 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:44:14.038 INFO:teuthology.orchestra.run.smithi189.stdout:Total 49 MB/s | 99 MB 00:02 2024-02-10T06:44:14.039 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-02-10T06:44:14.442 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-02-10T06:44:14.442 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-02-10T06:44:15.355 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-02-10T06:44:15.459 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-02-10T06:44:15.460 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-02-10T06:44:16.002 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:44:16.138 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:44:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-02-10T06:44:18.522 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:44:18.694 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:44:18.925 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:44:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:44:21.430 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:44:21.621 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:44:23.831 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:44:23.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:26.311 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:44:26.418 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:28.690 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:31.112 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:35.989 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 405 kB/s | 19 MB 00:47 2024-02-10T06:44:41.092 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 5.9 MB 00:01 2024-02-10T06:44:42.581 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 15 kB 00:00 2024-02-10T06:44:47.210 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 20 MB 00:04 2024-02-10T06:44:51.957 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-02-10T06:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T06:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T06:44:54.274 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:44:54.280 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout:Install 4 Packages 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 99 M 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 122 M 2024-02-10T06:44:54.281 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-02-10T06:44:54.625 INFO:teuthology.orchestra.run.smithi136.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 17 MB/s | 6.0 MB 00:00 2024-02-10T06:44:55.150 INFO:teuthology.orchestra.run.smithi136.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-02-10T06:44:55.783 INFO:teuthology.orchestra.run.smithi136.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 29 MB/s | 33 MB 00:01 2024-02-10T06:44:56.135 INFO:teuthology.orchestra.run.smithi136.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-10T06:44:56.137 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:44:56.137 INFO:teuthology.orchestra.run.smithi136.stdout:Total 53 MB/s | 99 MB 00:01 2024-02-10T06:44:56.137 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-02-10T06:44:56.533 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-02-10T06:44:56.533 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-02-10T06:44:57.570 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-02-10T06:44:57.571 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-02-10T06:44:57.900 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:57.900 INFO:teuthology.orchestra.run.smithi076.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T06:44:57.900 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:57.970 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:58.380 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:58.380 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:44:58.380 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:44:58.383 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:58.778 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-02-10T06:44:59.329 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:44:59.457 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:44:59.458 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:44:59.458 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T06:44:59.458 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T06:44:59.458 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T06:44:59.458 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-10T06:44:59.459 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T06:44:59.459 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:44:59.566 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:44:59.566 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:44:59.566 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T06:44:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:44:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T06:44:59.611 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T06:44:59.611 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-419.el9.x86_64 2024-02-10T06:44:59.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T06:44:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-02-10T06:45:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-02-10T06:45:00.405 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:00.405 INFO:teuthology.orchestra.run.smithi189.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T06:45:00.405 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:45:00.474 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:00.694 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:45:00.845 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:45:00.937 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:00.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:45:00.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:45:00.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:45:01.355 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:01.355 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:45:01.355 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-02-10T06:45:01.355 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:01.355 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T06:45:01.356 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T06:45:01.356 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T06:45:01.356 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:45:01.356 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-02-10T06:45:01.583 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T06:45:01.632 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-02-10T06:45:01.633 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-02-10T06:45:01.658 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.dC2hyx3PHY 2024-02-10T06:45:01.658 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dC2hyx3PHY 2024-02-10T06:45:01.728 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.dC2hyx3PHY 2024-02-10T06:45:01.862 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:45:01.863 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.dC2hyx3PHY is 6KB 2024-02-10T06:45:01.912 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.dC2hyx3PHY 2024-02-10T06:45:01.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T06:45:01.986 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:01.987 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:45:01.987 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T06:45:01.987 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T06:45:01.987 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T06:45:01.988 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-10T06:45:01.988 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T06:45:01.988 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:45:01.993 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T06:45:01.993 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T06:45:01.993 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T06:45:01.994 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T06:45:01.995 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T06:45:02.095 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:02.095 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:45:02.095 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 68.el9 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8264254 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T06:45:02.137 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:45:02.138 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T06:45:02.139 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T06:45:02.139 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 5.14.0-419.el9.x86_64 2024-02-10T06:45:02.139 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T06:45:02.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-02-10T06:45:02.732 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-02-10T06:45:03.606 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:45:03.791 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:45:04.129 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T06:45:04.169 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-02-10T06:45:04.171 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-02-10T06:45:04.187 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.s9UU7P8oAe 2024-02-10T06:45:04.187 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.s9UU7P8oAe 2024-02-10T06:45:04.255 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.s9UU7P8oAe 2024-02-10T06:45:04.397 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.s9UU7P8oAe is 6KB 2024-02-10T06:45:04.446 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.s9UU7P8oAe 2024-02-10T06:45:04.460 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T06:45:04.524 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T06:45:04.525 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T06:45:04.525 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T06:45:04.525 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T06:45:04.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T06:45:04.653 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-02-10T06:45:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:45:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:13.469 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:32.161 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-10T06:45:32.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:45:32.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:45:34.656 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-10T06:45:34.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:45:34.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:45:42.933 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:42.933 INFO:teuthology.orchestra.run.smithi136.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T06:45:42.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:45:43.001 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:45:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:45:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:45:43.890 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-02-10T06:45:44.406 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:45:44.530 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:44.530 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:45:44.531 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T06:45:44.531 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T06:45:44.531 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T06:45:44.531 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-10T06:45:44.531 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T06:45:44.531 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:45:44.639 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:45:44.640 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:45:44.640 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.06 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 68.el9 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Group : Unspecified 2024-02-10T06:45:44.683 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 8264254 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : builder@centos.org 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:45:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T06:45:44.685 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T06:45:44.685 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T06:45:44.685 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2024-02-10T06:45:44.685 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:45:44.685 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T06:45:44.686 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T06:45:44.686 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 5.14.0-419.el9.x86_64 2024-02-10T06:45:44.686 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T06:45:45.314 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-02-10T06:45:46.739 INFO:teuthology.orchestra.run.smithi136.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T06:45:46.771 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-02-10T06:45:46.780 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-02-10T06:45:46.798 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.KFJ3AtTsSe 2024-02-10T06:45:46.798 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KFJ3AtTsSe 2024-02-10T06:45:46.876 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.KFJ3AtTsSe 2024-02-10T06:45:47.023 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.KFJ3AtTsSe is 6KB 2024-02-10T06:45:47.072 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.KFJ3AtTsSe 2024-02-10T06:45:47.086 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T06:45:47.151 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T06:45:47.151 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T06:45:47.152 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T06:45:47.152 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T06:45:47.153 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T06:45:47.321 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-02-10T06:45:50.524 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-02-10T06:45:53.084 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-02-10T06:45:59.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:45:59.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:02.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:46:02.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:02.592 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-02-10T06:46:05.148 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-02-10T06:46:14.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:46:14.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:17.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:46:17.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:17.324 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-10T06:46:17.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:46:17.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:17.660 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-02-10T06:46:32.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:46:32.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:35.580 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-02-10T06:46:35.836 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-02-10T06:46:44.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:46:44.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:47.900 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-02-10T06:46:50.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:46:50.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:46:50.955 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-02-10T06:46:51.364 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:46:51.364 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-419.el9.x86_64"... 2024-02-10T06:46:51.364 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-02-10T06:46:51.378 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-419.el9.x86_64 2024-02-10T06:46:51.379 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T06:46:51.379 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T06:46:51.379 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T06:46:52.380 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-10T06:46:52.380 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T06:46:52.440 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-02-10T06:46:52.479 DEBUG:teuthology.parallel:result is None 2024-02-10T06:46:59.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:46:59.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:47:04.522 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-02-10T06:47:05.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:47:05.129 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-10T06:47:05.129 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-02-10T06:47:05.151 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-419.el9.x86_64 2024-02-10T06:47:05.151 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T06:47:05.151 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T06:47:05.151 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T06:47:06.152 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-10T06:47:06.152 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T06:47:06.211 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-02-10T06:47:06.265 DEBUG:teuthology.parallel:result is None 2024-02-10T06:47:18.336 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-02-10T06:47:33.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:47:33.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:47:33.771 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-02-10T06:47:34.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:47:34.313 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-10T06:47:34.314 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-02-10T06:47:34.337 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-419.el9.x86_64 2024-02-10T06:47:34.337 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T06:47:34.337 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T06:47:34.337 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T06:47:35.338 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-10T06:47:35.338 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T06:47:35.469 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-02-10T06:47:35.692 DEBUG:teuthology.parallel:result is None 2024-02-10T06:47:35.692 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-10T06:47:35.701 INFO:teuthology.task.internal:Creating test directory... 2024-02-10T06:47:35.701 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T06:47:35.704 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T06:47:35.707 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T06:47:35.729 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-10T06:47:35.735 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-10T06:47:35.747 INFO:teuthology.task.internal:Creating archive directory... 2024-02-10T06:47:35.754 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T06:47:35.762 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T06:47:35.764 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T06:47:35.837 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-10T06:47:35.843 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-10T06:47:35.844 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T06:47:35.846 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T06:47:35.848 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T06:47:35.878 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.878 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.886 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.890 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.893 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.900 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:47:35.902 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-10T06:47:35.909 INFO:teuthology.task.internal:Configuring sudo... 2024-02-10T06:47:35.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T06:47:35.938 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T06:47:35.941 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T06:47:35.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-10T06:47:35.981 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-10T06:47:35.981 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T06:47:36.009 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T06:47:36.011 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T06:47:36.028 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.156 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.239 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.283 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:47:36.284 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T06:47:36.352 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.395 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.461 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.544 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:47:36.545 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T06:47:36.612 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.657 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:47:36.719 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.762 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:47:36.806 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:47:36.806 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T06:47:36.877 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-02-10T06:47:36.879 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-02-10T06:47:36.881 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-02-10T06:47:36.917 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:47:36.927 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:47:36.961 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:47:37.418 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-10T06:47:37.425 INFO:teuthology.task.internal:Starting timer... 2024-02-10T06:47:37.425 INFO:teuthology.run_tasks:Running task pcp... 2024-02-10T06:47:37.441 INFO:teuthology.run_tasks:Running task selinux... 2024-02-10T06:47:37.449 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-10T06:47:37.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-02-10T06:47:37.507 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-02-10T06:47:37.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-02-10T06:47:37.576 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-02-10T06:47:37.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-02-10T06:47:37.630 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-02-10T06:47:37.632 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-10T06:47:37.632 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-02-10T06:47:37.652 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-02-10T06:47:37.653 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-02-10T06:47:37.693 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-02-10T06:47:37.693 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-02-10T06:47:37.736 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-02-10T06:47:37.737 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-02-10T06:47:37.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:47:37.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:47:37.768 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:47:37.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:47:37.797 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:47:37.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:47:37.828 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-10T06:47:37.828 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-02-10T06:47:37.871 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-02-10T06:47:37.903 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-02-10T06:47:37.938 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-10T06:47:37.947 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-10T06:47:37.966 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-10T06:47:38.086 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-10T06:47:38.105 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-10T06:47:38.137 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-10T06:51:59.115 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-02-10T06:51:59.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:51:59.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:51:59.217 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-02-10T06:51:59.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-10T06:51:59.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:51:59.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:51:59.389 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-02-10T06:51:59.464 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-02-10T06:51:59.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:51:59.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:51:59.558 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-02-10T06:51:59.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-10T06:51:59.635 INFO:teuthology.run_tasks:Running task clock... 2024-02-10T06:51:59.646 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-10T06:51:59.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T06:51:59.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:51:59.650 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T06:51:59.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:51:59.652 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T06:51:59.653 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:51:59.682 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T06:51:59.684 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T06:51:59.701 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T06:51:59.702 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T06:51:59.705 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T06:51:59.721 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T06:51:59.731 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-02-10T06:51:59.733 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-02-10T06:51:59.746 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-02-10T06:51:59.747 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-02-10T06:51:59.751 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-02-10T06:51:59.763 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T06:51:59.763 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T06:51:59.765 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-02-10T06:51:59.779 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T06:51:59.780 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T06:51:59.781 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T06:51:59.797 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T06:51:59.833 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.837 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.844 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:51:59.847 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:51:59.847 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-02-10T06:51:59.847 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.847 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.848 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.848 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.848 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.852 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:51:59.853 INFO:teuthology.run_tasks:Running task pexec... 2024-02-10T06:51:59.862 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-10T06:51:59.862 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi076.front.sepia.ceph.com 2024-02-10T06:51:59.862 DEBUG:teuthology.orchestra.run.smithi076:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T06:51:59.863 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi136.front.sepia.ceph.com 2024-02-10T06:51:59.863 DEBUG:teuthology.orchestra.run.smithi136:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T06:51:59.863 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi189.front.sepia.ceph.com 2024-02-10T06:51:59.863 DEBUG:teuthology.orchestra.run.smithi189:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T06:52:00.483 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:47 ago on Sat 10 Feb 2024 06:51:13 AM UTC. 2024-02-10T06:52:00.489 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:55 ago on Sat 10 Feb 2024 06:51:05 AM UTC. 2024-02-10T06:52:00.504 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:55 ago on Sat 10 Feb 2024 06:51:05 AM UTC. 2024-02-10T06:52:00.638 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-02-10T06:52:00.639 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 3.1 M 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 9.8 M 2024-02-10T06:52:00.640 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-02-10T06:52:00.642 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-02-10T06:52:00.659 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-02-10T06:52:00.660 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-02-10T06:52:00.661 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:52:00.662 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 3.1 M 2024-02-10T06:52:00.662 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 9.8 M 2024-02-10T06:52:00.662 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-02-10T06:52:00.662 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 3.1 M 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 9.8 M 2024-02-10T06:52:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-02-10T06:52:00.863 INFO:teuthology.orchestra.run.smithi136.stdout:runc-1.1.12-1.el9.x86_64.rpm 14 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.863 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:52:00.864 INFO:teuthology.orchestra.run.smithi136.stdout:Total 14 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.864 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-02-10T06:52:00.869 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-02-10T06:52:00.869 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-02-10T06:52:00.877 INFO:teuthology.orchestra.run.smithi189.stdout:runc-1.1.12-1.el9.x86_64.rpm 14 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.878 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:52:00.878 INFO:teuthology.orchestra.run.smithi189.stdout:Total 14 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.879 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-02-10T06:52:00.884 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-02-10T06:52:00.884 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-02-10T06:52:00.903 INFO:teuthology.orchestra.run.smithi076.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.903 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:52:00.903 INFO:teuthology.orchestra.run.smithi076.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-10T06:52:00.903 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-02-10T06:52:00.909 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-02-10T06:52:00.909 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-02-10T06:52:00.940 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-02-10T06:52:00.940 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-02-10T06:52:00.955 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-02-10T06:52:00.955 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-02-10T06:52:00.980 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-02-10T06:52:00.981 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-02-10T06:52:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-02-10T06:52:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-02-10T06:52:01.182 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-02-10T06:52:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:01.264 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:01.281 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:01.779 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:01.795 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:01.849 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:02.094 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:02.094 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:52:02.094 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-02-10T06:52:02.095 INFO:teuthology.orchestra.run.smithi189.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T06:52:02.095 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:52:02.095 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-02-10T06:52:02.095 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:02.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:52:02.096 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-02-10T06:52:02.096 INFO:teuthology.orchestra.run.smithi136.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T06:52:02.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-02-10T06:52:02.096 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-02-10T06:52:02.227 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:52:02.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:52:02.228 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-02-10T06:52:02.228 INFO:teuthology.orchestra.run.smithi076.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T06:52:02.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:52:02.228 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-02-10T06:52:02.328 DEBUG:teuthology.parallel:result is None 2024-02-10T06:52:02.348 DEBUG:teuthology.parallel:result is None 2024-02-10T06:52:02.499 DEBUG:teuthology.parallel:result is None 2024-02-10T06:52:02.499 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-10T06:52:02.509 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-10T06:52:02.510 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:52:02.510 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:02.525 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T06:52:02.525 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-02-10T06:52:02.579 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:51:58.038636327 +0000 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:51:32.396438893 +0000 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:51:32.396438893 +0000 2024-02-10T06:52:02.580 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:51:32.396438893 +0000 2024-02-10T06:52:02.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T06:52:02.645 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:52:02.645 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:52:02.645 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000267528 s, 1.9 MB/s 2024-02-10T06:52:02.647 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T06:52:02.702 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-02-10T06:52:02.757 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T06:52:02.757 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:51:58.038636327 +0000 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:51:32.726428565 +0000 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:51:32.726428565 +0000 2024-02-10T06:52:02.758 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:51:32.726428565 +0000 2024-02-10T06:52:02.758 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T06:52:02.822 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:52:02.823 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:52:02.823 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000183205 s, 2.8 MB/s 2024-02-10T06:52:02.825 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T06:52:02.880 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:51:58.038636327 +0000 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:51:33.057418205 +0000 2024-02-10T06:52:02.936 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:51:33.057418205 +0000 2024-02-10T06:52:02.937 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:51:33.057418205 +0000 2024-02-10T06:52:02.937 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T06:52:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:52:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:52:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000184751 s, 2.8 MB/s 2024-02-10T06:52:03.023 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T06:52:03.038 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:51:58.038636327 +0000 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:51:33.403407376 +0000 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:51:33.403407376 +0000 2024-02-10T06:52:03.093 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:51:33.403407376 +0000 2024-02-10T06:52:03.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T06:52:03.158 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:52:03.159 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:52:03.159 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000200288 s, 2.6 MB/s 2024-02-10T06:52:03.160 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T06:52:03.215 DEBUG:teuthology.orchestra.run.smithi076:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T06:52:03.353 INFO:teuthology.orchestra.run.smithi076.stdout:loop 2024-02-10T06:52:03.355 INFO:tasks.nvme_loop:Connecting nvme_loop smithi076:/dev/vg_nvme/lv_1... 2024-02-10T06:52:03.355 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T06:52:03.392 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-02-10T06:52:03.435 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/vg_nvme/lv_11 2024-02-10T06:52:03.466 INFO:tasks.nvme_loop:Connecting nvme_loop smithi076:/dev/vg_nvme/lv_2... 2024-02-10T06:52:03.467 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T06:52:03.545 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-02-10T06:52:03.583 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/vg_nvme/lv_21 2024-02-10T06:52:03.612 INFO:tasks.nvme_loop:Connecting nvme_loop smithi076:/dev/vg_nvme/lv_3... 2024-02-10T06:52:03.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T06:52:03.692 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-02-10T06:52:03.731 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/vg_nvme/lv_31 2024-02-10T06:52:03.763 INFO:tasks.nvme_loop:Connecting nvme_loop smithi076:/dev/vg_nvme/lv_4... 2024-02-10T06:52:03.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T06:52:03.852 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-02-10T06:52:03.891 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/vg_nvme/lv_41 2024-02-10T06:52:03.928 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:52:03.928 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:03.943 DEBUG:teuthology.orchestra.run.smithi076:> sudo nvme list 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/nvme0n1 PHFT620400U4400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/nvme1n1 0bbd64bcd61d913e7401 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/nvme2n1 f0744870f4d973221029 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/nvme3n1 39ac285e0f8e70e9b2c6 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:04.009 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/nvme4n1 8c456841778157842c71 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:04.010 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:52:04.010 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:52:04.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/scratch_devs 2024-02-10T06:52:04.077 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:52:04.077 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:04.096 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T06:52:04.096 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:51:58.052822065 +0000 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:51:36.527309767 +0000 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:51:36.527309767 +0000 2024-02-10T06:52:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:51:36.527309767 +0000 2024-02-10T06:52:04.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T06:52:04.222 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:52:04.223 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:52:04.223 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000225324 s, 2.3 MB/s 2024-02-10T06:52:04.224 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T06:52:04.282 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:51:58.052822065 +0000 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:51:36.874301893 +0000 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:51:36.874301893 +0000 2024-02-10T06:52:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:51:36.874301893 +0000 2024-02-10T06:52:04.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T06:52:04.407 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:52:04.407 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:52:04.407 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000207759 s, 2.5 MB/s 2024-02-10T06:52:04.408 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T06:52:04.465 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:51:58.052822065 +0000 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:51:37.228293860 +0000 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:51:37.228293860 +0000 2024-02-10T06:52:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:51:37.228293860 +0000 2024-02-10T06:52:04.522 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T06:52:04.588 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:52:04.588 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:52:04.588 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000209503 s, 2.4 MB/s 2024-02-10T06:52:04.589 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T06:52:04.645 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:51:58.052822065 +0000 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:51:37.554286463 +0000 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:51:37.554286463 +0000 2024-02-10T06:52:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:51:37.554286463 +0000 2024-02-10T06:52:04.701 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T06:52:04.768 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:52:04.768 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:52:04.768 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000184752 s, 2.8 MB/s 2024-02-10T06:52:04.769 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T06:52:04.825 DEBUG:teuthology.orchestra.run.smithi136:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T06:52:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:loop 2024-02-10T06:52:04.963 INFO:tasks.nvme_loop:Connecting nvme_loop smithi136:/dev/vg_nvme/lv_1... 2024-02-10T06:52:04.963 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T06:52:05.000 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-02-10T06:52:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/vg_nvme/lv_11 2024-02-10T06:52:05.088 INFO:tasks.nvme_loop:Connecting nvme_loop smithi136:/dev/vg_nvme/lv_2... 2024-02-10T06:52:05.089 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T06:52:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-02-10T06:52:05.208 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/vg_nvme/lv_21 2024-02-10T06:52:05.237 INFO:tasks.nvme_loop:Connecting nvme_loop smithi136:/dev/vg_nvme/lv_3... 2024-02-10T06:52:05.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T06:52:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-02-10T06:52:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/vg_nvme/lv_31 2024-02-10T06:52:05.386 INFO:tasks.nvme_loop:Connecting nvme_loop smithi136:/dev/vg_nvme/lv_4... 2024-02-10T06:52:05.386 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T06:52:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-02-10T06:52:05.505 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/vg_nvme/lv_41 2024-02-10T06:52:05.534 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:52:05.534 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:05.597 DEBUG:teuthology.orchestra.run.smithi136:> sudo nvme list 2024-02-10T06:52:05.663 INFO:teuthology.orchestra.run.smithi136.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T06:52:05.663 INFO:teuthology.orchestra.run.smithi136.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T06:52:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/nvme0n1 CVPD514100122P0SGN INTEL SSDPEDMX020T4 1 2.00 TB / 2.00 TB 512 B + 0 B 8DV10171 2024-02-10T06:52:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/nvme1n1 fb47b372b904fcaad8a7 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/nvme2n1 a57f22516f2564befdc8 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/nvme3n1 28930fd89c0cba5f79c3 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:/dev/nvme4n1 eeb92fdfa6799b14ad7b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:05.665 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:52:05.665 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:52:05.665 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/scratch_devs 2024-02-10T06:52:05.732 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:52:05.732 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:05.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T06:52:05.750 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-02-10T06:52:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T06:52:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:05.811 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-10T06:52:05.811 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:05.812 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:05.812 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:51:58.074549116 +0000 2024-02-10T06:52:05.812 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:51:30.524088033 +0000 2024-02-10T06:52:05.812 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:51:30.524088033 +0000 2024-02-10T06:52:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:51:30.524088033 +0000 2024-02-10T06:52:05.812 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T06:52:05.879 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:52:05.879 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:52:05.879 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000240827 s, 2.1 MB/s 2024-02-10T06:52:05.881 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T06:52:05.939 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-02-10T06:52:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T06:52:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:05.997 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-10T06:52:05.997 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:05.997 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:05.998 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:51:58.074549116 +0000 2024-02-10T06:52:05.998 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:51:30.884080991 +0000 2024-02-10T06:52:05.998 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:51:30.884080991 +0000 2024-02-10T06:52:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:51:30.884080991 +0000 2024-02-10T06:52:05.998 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T06:52:06.067 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:52:06.068 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:52:06.068 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000231199 s, 2.2 MB/s 2024-02-10T06:52:06.069 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T06:52:06.128 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-02-10T06:52:06.185 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T06:52:06.185 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:06.185 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-10T06:52:06.185 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:06.186 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:06.186 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:51:58.074549116 +0000 2024-02-10T06:52:06.186 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:51:31.221074400 +0000 2024-02-10T06:52:06.186 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:51:31.221074400 +0000 2024-02-10T06:52:06.186 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:51:31.221074400 +0000 2024-02-10T06:52:06.187 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T06:52:06.253 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:52:06.254 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:52:06.254 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000234574 s, 2.2 MB/s 2024-02-10T06:52:06.255 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T06:52:06.313 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:51:58.074549116 +0000 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:51:31.578067417 +0000 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:51:31.578067417 +0000 2024-02-10T06:52:06.370 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:51:31.578067417 +0000 2024-02-10T06:52:06.370 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T06:52:06.437 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:52:06.438 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:52:06.438 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000213171 s, 2.4 MB/s 2024-02-10T06:52:06.439 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T06:52:06.496 DEBUG:teuthology.orchestra.run.smithi189:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T06:52:06.635 INFO:teuthology.orchestra.run.smithi189.stdout:loop 2024-02-10T06:52:06.636 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_1... 2024-02-10T06:52:06.636 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T06:52:06.674 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-02-10T06:52:06.712 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_11 2024-02-10T06:52:06.749 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_2... 2024-02-10T06:52:06.749 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T06:52:06.836 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-02-10T06:52:06.876 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_21 2024-02-10T06:52:06.913 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_3... 2024-02-10T06:52:06.913 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T06:52:06.952 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-02-10T06:52:06.991 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_31 2024-02-10T06:52:07.020 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_4... 2024-02-10T06:52:07.021 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T06:52:07.099 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-02-10T06:52:07.139 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_41 2024-02-10T06:52:07.170 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:52:07.170 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:52:07.225 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme list 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/nvme0n1 CVFT6232004L400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/nvme1n1 d4e4981338ece477f21d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/nvme2n1 9068d91efb19b7abd9cd Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/nvme3n1 e78950c98353815517c4 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:07.291 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/nvme4n1 83a9f29c9023692046fc Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T06:52:07.292 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:52:07.292 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:52:07.292 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/scratch_devs 2024-02-10T06:52:07.359 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-10T06:52:07.452 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'} 2024-02-10T06:52:07.452 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:52:07.452 INFO:tasks.cephadm:Cluster fsid is e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:52:07.452 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-10T06:52:07.453 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.76', 'mon.b': '172.21.15.136', 'mon.c': '172.21.15.189'} 2024-02-10T06:52:07.453 INFO:tasks.cephadm:First mon is mon.a on smithi076 2024-02-10T06:52:07.453 INFO:tasks.cephadm:First mgr is y 2024-02-10T06:52:07.453 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-10T06:52:07.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-02-10T06:52:07.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-02-10T06:52:07.519 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-02-10T06:52:07.547 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-10T06:52:07.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:52:07.814 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76638/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1361-g290212a1', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1361.g290212a1'}, 'url': 'https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-09 18:34:21.832169', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-10T06:52:07.988 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref main, sha1 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:52:07.989 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm 2024-02-10T06:52:07.990 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:08.814 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 06:52 /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:08.815 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:09.409 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 06:52 /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:09.410 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:10.095 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 06:52 /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:10.095 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:10.114 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:10.133 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T06:52:10.165 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 on all hosts... 2024-02-10T06:52:10.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T06:52:10.167 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T06:52:10.176 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T06:52:10.945 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:52:11.013 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:52:11.032 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-02-10T06:52:40.160 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-02-10T06:52:58.624 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-02-10T06:53:01.014 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-10T06:53:01.014 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:53:01.014 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T06:53:01.014 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-02-10T06:53:01.015 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T06:53:01.015 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-02-10T06:53:01.015 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-10T06:53:01.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-02-10T06:53:01.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-02-10T06:53:01.109 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-02-10T06:53:01.139 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-02-10T06:53:01.168 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-02-10T06:53:01.198 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-02-10T06:53:01.225 INFO:tasks.cephadm:Writing seed config... 2024-02-10T06:53:01.226 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-10T06:53:01.226 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-10T06:53:01.226 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-10T06:53:01.226 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-10T06:53:01.226 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-10T06:53:01.227 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-10T06:53:01.227 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-10T06:53:01.227 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-10T06:53:01.227 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-10T06:53:01.227 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-10T06:53:01.227 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:53:01.227 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-10T06:53:01.244 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e8085918-c7e0-11ee-95b8-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-10T06:53:01.245 DEBUG:teuthology.orchestra.run.smithi076:mon.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service 2024-02-10T06:53:01.287 DEBUG:teuthology.orchestra.run.smithi076:mgr.y> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y.service 2024-02-10T06:53:01.329 INFO:tasks.cephadm:Bootstrapping... 2024-02-10T06:53:01.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -v bootstrap --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.76 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:53:01.521 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:53:01.521 INFO:teuthology.orchestra.run.smithi076.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', '-v', 'bootstrap', '--fsid', 'e8085918-c7e0-11ee-95b8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.76', '--skip-admin-label'] 2024-02-10T06:53:01.547 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:53:01.548 INFO:teuthology.orchestra.run.smithi076.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-10T06:53:01.548 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying podman|docker is present... 2024-02-10T06:53:01.571 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:53:01.571 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying lvm2 is present... 2024-02-10T06:53:01.571 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying time synchronization is in place... 2024-02-10T06:53:01.578 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T06:53:01.578 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T06:53:01.584 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T06:53:01.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:53:01.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout enabled 2024-02-10T06:53:01.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout active 2024-02-10T06:53:01.596 INFO:teuthology.orchestra.run.smithi076.stdout:Unit chronyd.service is enabled and running 2024-02-10T06:53:01.596 INFO:teuthology.orchestra.run.smithi076.stdout:Repeating the final host check... 2024-02-10T06:53:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:53:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-10T06:53:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl is present 2024-02-10T06:53:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:lvcreate is present 2024-02-10T06:53:01.626 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T06:53:01.626 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T06:53:01.632 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T06:53:01.632 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:53:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout enabled 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout active 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:Unit chronyd.service is enabled and running 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:Host looks OK 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:Cluster fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:Acquiring lock 140126432799904 on /run/cephadm/e8085918-c7e0-11ee-95b8-87774f69a715.lock 2024-02-10T06:53:01.644 INFO:teuthology.orchestra.run.smithi076.stdout:Lock 140126432799904 acquired on /run/cephadm/e8085918-c7e0-11ee-95b8-87774f69a715.lock 2024-02-10T06:53:01.645 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying IP 172.21.15.76 port 3300 ... 2024-02-10T06:53:01.645 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying IP 172.21.15.76 port 6789 ... 2024-02-10T06:53:01.645 INFO:teuthology.orchestra.run.smithi076.stdout:Base mon IP(s) is [172.21.15.76:3300, 172.21.15.76:6789], mon addrv is [v2:172.21.15.76:3300,v1:172.21.15.76:6789] 2024-02-10T06:53:01.648 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.76 metric 100 2024-02-10T06:53:01.648 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.76 metric 100 2024-02-10T06:53:01.651 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-10T06:53:01.651 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-10T06:53:01.651 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1717sec hoplimit 64 pref medium 2024-02-10T06:53:01.653 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:Mon IP `172.21.15.76` is in CIDR network `172.21.0.0/20` 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:Mon IP `172.21.15.76` is in CIDR network `172.21.0.0/20` 2024-02-10T06:53:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-10T06:53:01.655 INFO:teuthology.orchestra.run.smithi076.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-10T06:53:01.655 INFO:teuthology.orchestra.run.smithi076.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:53:02.083 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Getting image source signatures 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying blob sha256:5fe0e97a267976e17c7dcb240cf9455472de6876101ae78af4bdf182158b6d8d 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying config sha256:c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T06:53:02.084 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-10T06:53:03.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph: stdout ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T06:53:03.506 INFO:teuthology.orchestra.run.smithi076.stdout:Ceph version: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T06:53:03.506 INFO:teuthology.orchestra.run.smithi076.stdout:Extracting ceph user uid/gid from container image... 2024-02-10T06:53:04.511 INFO:teuthology.orchestra.run.smithi076.stdout:stat: stdout 167 167 2024-02-10T06:53:04.511 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial keys... 2024-02-10T06:53:05.424 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQBQHcdlQE54MhAAq/6LHmIUW+YYlwAB8hvyHA== 2024-02-10T06:53:06.371 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQBRHcdlGGFRLRAAxK0nWDv+iTDaNchFWpAKvw== 2024-02-10T06:53:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQBSHcdlGcMmKxAAwtTHvc3OX98/hbRCUR9HIA== 2024-02-10T06:53:07.328 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial monmap... 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool for a [v2:172.21.15.76:3300,v1:172.21.15.76:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-02-10T06:53:08.150 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: set fsid to e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:08.151 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T06:53:08.151 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:53:08.151 INFO:teuthology.orchestra.run.smithi076.stdout:Creating mon... 2024-02-10T06:53:09.030 INFO:teuthology.orchestra.run.smithi076.stdout:create mon.a on 2024-02-10T06:53:09.379 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-10T06:53:09.549 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target. 2024-02-10T06:53:09.549 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target. 2024-02-10T06:53:09.783 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a 2024-02-10T06:53:09.783 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service: Unit ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service not loaded. 2024-02-10T06:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target.wants/ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715@.service. 2024-02-10T06:53:10.454 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:53:10.454 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-02-10T06:53:10.460 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:53:10.461 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:53:10.461 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-02-10T06:53:10.461 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T06:53:10.461 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mon to start... 2024-02-10T06:53:10.461 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mon... 2024-02-10T06:53:11.530 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:11 smithi076 ceph-mon[25215]: from='client.? 172.21.15.76:0/2744218996' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-10T06:53:11.533 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout cluster: 2024-02-10T06:53:11.534 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout id: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:11.534 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout services: 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558876s) 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout data: 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout pgs: 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:mon is available 2024-02-10T06:53:11.535 INFO:teuthology.orchestra.run.smithi076.stdout:Assimilating anything we can from ceph.conf... 2024-02-10T06:53:12.612 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:12 smithi076 ceph-mon[25215]: from='client.? 172.21.15.76:0/1058446394' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T06:53:12.612 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:12 smithi076 ceph-mon[25215]: from='client.? 172.21.15.76:0/1058446394' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T06:53:12.615 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout fsid = e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.76:3300,v1:172.21.15.76:6789] 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:12.616 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T06:53:12.617 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T06:53:12.617 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T06:53:12.617 INFO:teuthology.orchestra.run.smithi076.stdout:Generating new minimal ceph.conf... 2024-02-10T06:53:13.630 INFO:teuthology.orchestra.run.smithi076.stdout:Restarting the monitor... 2024-02-10T06:53:13.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:13 smithi076 ceph-mon[25215]: from='client.? 172.21.15.76:0/1939582113' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:53:13.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:13 smithi076 systemd[1]: Stopping Ceph mon.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:53:14.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:13 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a[25192]: 2024-02-10T06:53:13.940+0000 7fdb50fee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:53:14.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:13 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a[25192]: 2024-02-10T06:53:13.940+0000 7fdb50fee700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-10T06:53:14.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 podman[25548]: 2024-02-10 06:53:14.186151903 +0000 UTC m=+0.398706151 container died bcb6237b5857e03a4ccdef6eeab0afc72e515823029571f220f070fbad547a17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-10T06:53:14.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 podman[25548]: 2024-02-10 06:53:14.546111871 +0000 UTC m=+0.758666125 container cleanup bcb6237b5857e03a4ccdef6eeab0afc72e515823029571f220f070fbad547a17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-10T06:53:14.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 bash[25548]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a 2024-02-10T06:53:15.156 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 podman[25567]: 2024-02-10 06:53:14.805129594 +0000 UTC m=+0.614000034 container remove bcb6237b5857e03a4ccdef6eeab0afc72e515823029571f220f070fbad547a17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=main-290212a) 2024-02-10T06:53:15.156 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-10T06:53:15.156 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 systemd[1]: Stopped Ceph mon.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:53:15.156 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:14 smithi076 systemd[1]: Starting Ceph mon.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:53:15.416 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 podman[25661]: 2024-02-10 06:53:15.156035869 +0000 UTC m=+0.103396860 container create dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=main-290212a, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 podman[25661]: 2024-02-10 06:53:15.070257465 +0000 UTC m=+0.017618429 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 podman[25661]: 2024-02-10 06:53:15.341520215 +0000 UTC m=+0.288881164 container init dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 podman[25661]: 2024-02-10 06:53:15.346732218 +0000 UTC m=+0.294093168 container start dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 8 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: pidfile_write: ignore empty --pid-file 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: load: jerasure load: lrc 2024-02-10T06:53:15.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: RocksDB version: 7.9.2 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Git sha 0 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: DB SUMMARY 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: DB Session ID: 5NEGHR2LFZXRCRQ7TS9E 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: CURRENT file: CURRENT 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84627 ; 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.error_if_exists: 0 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.create_if_missing: 0 2024-02-10T06:53:15.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.paranoid_checks: 1 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.env: 0x556f76153ae0 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.info_log: 0x556f774f98c0 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.statistics: (nil) 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.use_fsync: 0 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_log_file_size: 0 2024-02-10T06:53:15.419 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_fallocate: 1 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.use_direct_reads: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.db_log_dir: 2024-02-10T06:53:15.420 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.wal_dir: 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.write_buffer_manager: 0x556f774cd400 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T06:53:15.421 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T06:53:15.422 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T06:53:15.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T06:53:15.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T06:53:15.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.unordered_write: 0 2024-02-10T06:53:15.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T06:53:15.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.row_cache: None 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.wal_filter: None 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.two_write_queues: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.wal_compression: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.atomic_flush: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T06:53:15.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.log_readahead_size: 0 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_background_jobs: 2 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_background_compactions: -1 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_subcompactions: 1 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T06:53:15.425 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_open_files: -1 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_background_flushes: -1 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Compression algorithms supported: 2024-02-10T06:53:15.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kZSTD supported: 0 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kXpressCompression supported: 0 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kZlibCompression supported: 1 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kSnappyCompression supported: 1 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kLZ4Compression supported: 1 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: kBZip2Compression supported: 0 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.merge_operator: 2024-02-10T06:53:15.427 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_filter: None 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556f774f99c0) 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: cache_index_and_filter_blocks: 1 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: pin_top_level_index_and_filter: 1 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_type: 0 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: data_block_index_type: 0 2024-02-10T06:53:15.428 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_shortening: 1 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: checksum: 4 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: no_block_cache: 0 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache: 0x556f774fd090 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_name: BinnedLRUCache 2024-02-10T06:53:15.429 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_options: 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: capacity : 536870912 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: num_shard_bits : 4 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: strict_capacity_limit : 0 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: high_pri_pool_ratio: 0.000 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_compressed: (nil) 2024-02-10T06:53:15.430 INFO:journalctl@ceph.mon.a.smithi076.stdout: persistent_cache: (nil) 2024-02-10T06:53:15.431 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_size: 4096 2024-02-10T06:53:15.431 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_size_deviation: 10 2024-02-10T06:53:15.431 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_restart_interval: 16 2024-02-10T06:53:15.431 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_block_restart_interval: 1 2024-02-10T06:53:15.431 INFO:journalctl@ceph.mon.a.smithi076.stdout: metadata_block_size: 4096 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: partition_filters: 0 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: use_delta_encoding: 1 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: filter_policy: bloomfilter 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: whole_key_filtering: 1 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: verify_compression: 0 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: read_amp_bytes_per_bit: 0 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: format_version: 5 2024-02-10T06:53:15.432 INFO:journalctl@ceph.mon.a.smithi076.stdout: enable_index_compression: 1 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_align: 0 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout: max_auto_readahead_size: 262144 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout: prepopulate_block_cache: 0 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout: initial_auto_readahead_size: 8192 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T06:53:15.433 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression: NoCompression 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.num_levels: 7 2024-02-10T06:53:15.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T06:53:15.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T06:53:15.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T06:53:15.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T06:53:15.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T06:53:15.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:53:15.436 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:53:15.437 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T06:53:15.438 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T06:53:15.439 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T06:53:15.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T06:53:15.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T06:53:15.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T06:53:15.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T06:53:15.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T06:53:15.441 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T06:53:15.442 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T06:53:15.443 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T06:53:15.443 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T06:53:15.443 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T06:53:15.443 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T06:53:15.443 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.table_properties_collectors: 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.inplace_update_support: 0 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T06:53:15.444 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T06:53:15.445 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.bloom_locality: 0 2024-02-10T06:53:15.445 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.max_successive_merges: 0 2024-02-10T06:53:15.445 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T06:53:15.445 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T06:53:15.445 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.ttl: 2592000 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T06:53:15.446 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enable_blob_files: false 2024-02-10T06:53:15.447 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.min_blob_size: 0 2024-02-10T06:53:15.447 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T06:53:15.447 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T06:53:15.447 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T06:53:15.447 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-10T06:53:15.448 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-10T06:53:15.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 890c733b-9b2e-413c-8265-da25f3fa96b8 2024-02-10T06:53:15.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547995375431, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-10T06:53:15.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-10T06:53:15.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547995376281, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79439, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73748, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707547995, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "890c733b-9b2e-413c-8265-da25f3fa96b8", "db_session_id": "5NEGHR2LFZXRCRQ7TS9E", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-10T06:53:15.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547995376362, "job": 1, "event": "recovery_finished"} 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556f775ee000 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: DB pointer 0x556f775d8000 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T06:53:15.450 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** DB Stats ** 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:53:15.451 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T06:53:15.452 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:53:15.452 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:53:15.452 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.452 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** Compaction Stats [default] ** 2024-02-10T06:53:15.452 INFO:journalctl@ceph.mon.a.smithi076.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: L0 2/0 81.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: Sum 2/0 81.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.453 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** Compaction Stats [default] ** 2024-02-10T06:53:15.454 INFO:journalctl@ceph.mon.a.smithi076.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:53:15.454 INFO:journalctl@ceph.mon.a.smithi076.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T06:53:15.454 INFO:journalctl@ceph.mon.a.smithi076.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:53:15.454 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.454 INFO:journalctl@ceph.mon.a.smithi076.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T06:53:15.455 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative compaction: 0.00 GB write, 26.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval compaction: 0.00 GB write, 26.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: Block cache BinnedLRUCache@0x556f774fd090#8 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: starting mon.a rank 0 at public addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] at bind addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???) e1 preinit fsid e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).mds e1 new map 2024-02-10T06:53:15.456 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).mds e1 print_map 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: e1 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: legacy client fscid: -1 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout: No filesystems configured 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:53:15.457 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:53:15.458 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-10T06:53:15.458 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 bash[25661]: dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d 2024-02-10T06:53:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 systemd[1]: Started Ceph mon.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: monmap e1: 1 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: fsmap 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T06:53:15.680 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:15 smithi076 ceph-mon[25700]: mgrmap e1: no daemons active 2024-02-10T06:53:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-10T06:53:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:53:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:Creating mgr... 2024-02-10T06:53:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-10T06:53:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-10T06:53:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y 2024-02-10T06:53:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y.service: Unit ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y.service not loaded. 2024-02-10T06:53:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715.target.wants/ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-e8085918-c7e0-11ee-95b8-87774f69a715@.service. 2024-02-10T06:53:16.967 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:16 smithi076 systemd[1]: Starting Ceph mgr.y for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:53:17.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:16 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3951734910' entity='client.admin' 2024-02-10T06:53:17.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 podman[25941]: 2024-02-10 06:53:17.088446057 +0000 UTC m=+0.017463068 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:53:17.285 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 podman[25941]: 2024-02-10 06:53:17.193324554 +0000 UTC m=+0.122341606 container create 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-290212a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131) 2024-02-10T06:53:17.483 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:53:17.484 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-02-10T06:53:17.490 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:53:17.491 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:53:17.491 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-02-10T06:53:17.491 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T06:53:17.499 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:53:17.499 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr to start... 2024-02-10T06:53:17.506 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr... 2024-02-10T06:53:17.698 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 podman[25941]: 2024-02-10 06:53:17.404084493 +0000 UTC m=+0.333101503 container init 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, ceph=True) 2024-02-10T06:53:17.698 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 podman[25941]: 2024-02-10 06:53:17.40942578 +0000 UTC m=+0.338442791 container start 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y, RELEASE=main-290212a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:53:17.698 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 bash[25941]: 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 2024-02-10T06:53:17.698 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 systemd[1]: Started Ceph mgr.y for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:53:18.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:17 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3220236067' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:53:18.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:17 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:17.970+0000 7f43ec00b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:53:18.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:18 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:18.070+0000 7f43ec00b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "e8085918-c7e0-11ee-95b8-87774f69a715", 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:53:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:53:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:53:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:53:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:53:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:53:10.467355+0000", 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:mgr not available, waiting (1/15)... 2024-02-10T06:53:18.942 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:18 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:18.714+0000 7f43ec00b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:53:18.942 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:18 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:18.805+0000 7f43ec00b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:53:18.942 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:18 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:18.941+0000 7f43ec00b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:53:19.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.035+0000 7f43ec00b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:53:19.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.115+0000 7f43ec00b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:53:19.658 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.371+0000 7f43ec00b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:53:19.658 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.478+0000 7f43ec00b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:53:19.658 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.563+0000 7f43ec00b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:53:19.659 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.657+0000 7f43ec00b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:53:20.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:19 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:19.830+0000 7f43ec00b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:53:21.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:20 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:20.799+0000 7f43ec00b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:53:21.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:20 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:20.934+0000 7f43ec00b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:53:21.363 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:21 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:21.273+0000 7f43ec00b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:53:21.363 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:21 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:21.361+0000 7f43ec00b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:53:21.742 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:21 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/257336397' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:53:21.743 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:21 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:21.558+0000 7f43ec00b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:53:21.743 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:21 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:21.741+0000 7f43ec00b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:53:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "e8085918-c7e0-11ee-95b8-87774f69a715", 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:53:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:53:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:53:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:53:10.467355+0000", 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:mgr not available, waiting (2/15)... 2024-02-10T06:53:22.281 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:22 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:22.024+0000 7f43ec00b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:53:22.281 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:22 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:22.280+0000 7f43ec00b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:53:22.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:22 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:22.512+0000 7f43ec00b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:53:23.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:23 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:23.308+0000 7f43ec00b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:53:24.135 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:23 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:23.875+0000 7f43ec00b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:53:24.135 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:23 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:23.974+0000 7f43ec00b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:24.196+0000 7f43ec00b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: Activating manager daemon y 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: mgrmap e2: y(active, starting, since 0.00286093s) 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:53:24.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: Manager daemon y is now available 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' 2024-02-10T06:53:24.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:24 smithi076 ceph-mon[25700]: from='mgr.14100 172.21.15.76:0/3335811984' entity='mgr.y' 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "e8085918-c7e0-11ee-95b8-87774f69a715", 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:53:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:53:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:53:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:53:10.467355+0000", 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:mgr not available, waiting (3/15)... 2024-02-10T06:53:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:25 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/51213152' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:53:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:25 smithi076 ceph-mon[25700]: mgrmap e3: y(active, since 1.00694s) 2024-02-10T06:53:27.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:27 smithi076 ceph-mon[25700]: mgrmap e4: y(active, since 2s) 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "e8085918-c7e0-11ee-95b8-87774f69a715", 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:53:28.168 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:53:28.171 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:53:28.172 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:53:10.467355+0000", 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:28.173 INFO:teuthology.orchestra.run.smithi076.stdout:mgr is available 2024-02-10T06:53:28.426 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:28 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2451945199' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout fsid = e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T06:53:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T06:53:29.380 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling cephadm module... 2024-02-10T06:53:29.744 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:29 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3046978628' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T06:53:29.744 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:29 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3046978628' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T06:53:30.649 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:30 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/330487066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-10T06:53:30.650 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:30 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setuser ceph since I am not root 2024-02-10T06:53:30.650 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:30 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setgroup ceph since I am not root 2024-02-10T06:53:31.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:30 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:30.983+0000 7fd60b611200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:53:31.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:31.083+0000 7fd60b611200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:53:31.622 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/330487066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-10T06:53:31.622 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-mon[25700]: mgrmap e5: y(active, since 6s) 2024-02-10T06:53:31.984 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:31.750+0000 7fd60b611200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:53:31.984 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:31.843+0000 7fd60b611200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:53:31.985 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:31 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:31.984+0000 7fd60b611200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:53:32.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.083+0000 7fd60b611200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:53:32.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.166+0000 7fd60b611200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:53:32.340 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for the mgr to restart... 2024-02-10T06:53:32.341 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr epoch 5... 2024-02-10T06:53:32.606 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/436112539' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T06:53:32.606 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.439+0000 7fd60b611200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:53:32.606 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.550+0000 7fd60b611200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:53:32.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.640+0000 7fd60b611200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:53:32.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.736+0000 7fd60b611200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:53:33.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:32 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:32.917+0000 7fd60b611200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:53:34.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:33 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:33.870+0000 7fd60b611200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:53:34.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:34 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:34.000+0000 7fd60b611200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:53:34.614 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:34 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:34.330+0000 7fd60b611200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:53:34.615 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:34 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:34.420+0000 7fd60b611200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:53:34.615 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:34 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:34.614+0000 7fd60b611200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:53:35.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:34 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:34.797+0000 7fd60b611200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:53:35.335 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:35 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:35.078+0000 7fd60b611200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:53:35.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:35 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:35.334+0000 7fd60b611200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:53:35.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:35 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:35.567+0000 7fd60b611200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:53:36.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:36.362+0000 7fd60b611200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:53:37.241 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:36.927+0000 7fd60b611200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:53:37.241 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:37.025+0000 7fd60b611200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:53:37.241 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:37.240+0000 7fd60b611200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:53:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: Active manager daemon y restarted 2024-02-10T06:53:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: Activating manager daemon y 2024-02-10T06:53:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: osdmap e2: 0 total, 0 up, 0 in 2024-02-10T06:53:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: mgrmap e6: y(active, starting, since 0.0030229s) 2024-02-10T06:53:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: Manager daemon y is now available 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:37 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:38.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:38 smithi076 ceph-mon[25700]: Found migration_current of "None". Setting to last migration. 2024-02-10T06:53:38.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:38 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:38.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:38 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:53:38.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:38 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:53:38.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:38 smithi076 ceph-mon[25700]: mgrmap e7: y(active, since 1.00531s) 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:mgr epoch 5 is available 2024-02-10T06:53:38.838 INFO:teuthology.orchestra.run.smithi076.stdout:Setting orchestrator backend to cephadm... 2024-02-10T06:53:39.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:39 smithi076 ceph-mon[25700]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T06:53:39.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:39 smithi076 ceph-mon[25700]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T06:53:39.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:39 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:39.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:39 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:40.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:40 smithi076 ceph-mon[25700]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:40.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:40 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:40.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:40 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:40.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:40 smithi076 ceph-mon[25700]: mgrmap e8: y(active, since 2s) 2024-02-10T06:53:41.275 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-10T06:53:41.275 INFO:teuthology.orchestra.run.smithi076.stdout:Generating ssh key... 2024-02-10T06:53:41.686 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: [10/Feb/2024:06:53:40] ENGINE Bus STARTING 2024-02-10T06:53:41.686 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: [10/Feb/2024:06:53:40] ENGINE Serving on https://172.21.15.76:7150 2024-02-10T06:53:41.686 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: [10/Feb/2024:06:53:40] ENGINE Serving on http://172.21.15.76:8765 2024-02-10T06:53:41.686 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: [10/Feb/2024:06:53:40] ENGINE Bus STARTED 2024-02-10T06:53:41.686 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:41.687 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:41 smithi076 ceph-mon[25700]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: Generating public/private rsa key pair. 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: Your identification has been saved in /tmp/tmpviolfe5y/key. 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: Your public key has been saved in /tmp/tmpviolfe5y/key.pub. 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: The key fingerprint is: 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: SHA256:LSQfFvdB0DDIwwRtPoqea0i+Cf7glmCMYVBGAv7D6HI ceph-e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: The key's randomart image is: 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: +---[RSA 3072]----+ 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |+o+ .*o.==o | 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |oo Bo o.. | 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |.. .o+. . | 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |..+ =oo | 2024-02-10T06:53:42.412 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |+o + . .S.. | 2024-02-10T06:53:42.413 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |+o. o . . | 2024-02-10T06:53:42.413 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |==Eo . | 2024-02-10T06:53:42.413 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |+=+.+ | 2024-02-10T06:53:42.413 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: |.o++.. | 2024-02-10T06:53:42.413 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: +----[SHA256]-----+ 2024-02-10T06:53:43.343 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:43 smithi076 ceph-mon[25700]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:43.343 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:43 smithi076 ceph-mon[25700]: Generating ssh key... 2024-02-10T06:53:43.343 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:43 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:43.343 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:43 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:43.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:53:43.930 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-10T06:53:43.930 INFO:teuthology.orchestra.run.smithi076.stdout:Adding key to root@localhost authorized_keys... 2024-02-10T06:53:43.930 INFO:teuthology.orchestra.run.smithi076.stdout:Adding host smithi076... 2024-02-10T06:53:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:44 smithi076 ceph-mon[25700]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:46.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:46 smithi076 ceph-mon[25700]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "addr": "172.21.15.76", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:46.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:46 smithi076 ceph-mon[25700]: Deploying cephadm binary to smithi076 2024-02-10T06:53:49.453 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Added host 'smithi076' with addr '172.21.15.76' 2024-02-10T06:53:49.453 INFO:teuthology.orchestra.run.smithi076.stdout:Deploying unmanaged mon service... 2024-02-10T06:53:50.132 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:49 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:50.133 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:49 smithi076 ceph-mon[25700]: Added host smithi076 2024-02-10T06:53:50.133 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:49 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:50.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-10T06:53:50.897 INFO:teuthology.orchestra.run.smithi076.stdout:Deploying unmanaged mgr service... 2024-02-10T06:53:51.255 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:51 smithi076 ceph-mon[25700]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:51.255 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:51 smithi076 ceph-mon[25700]: Saving service mon spec with placement count:5 2024-02-10T06:53:51.255 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:51 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:53.007 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-10T06:53:53.128 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:52 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:53.128 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:52 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:53.129 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:52 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:54.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:53 smithi076 ceph-mon[25700]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:53:54.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:53 smithi076 ceph-mon[25700]: Saving service mgr spec with placement count:2 2024-02-10T06:53:54.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:53 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3202179079' entity='client.admin' 2024-02-10T06:53:55.847 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling the dashboard module... 2024-02-10T06:53:56.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:56 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:56.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:56 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3239118473' entity='client.admin' 2024-02-10T06:53:57.478 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/101455915' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-10T06:53:57.479 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-mon[25700]: from='mgr.14120 172.21.15.76:0/273909707' entity='mgr.y' 2024-02-10T06:53:57.479 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setuser ceph since I am not root 2024-02-10T06:53:57.479 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setgroup ceph since I am not root 2024-02-10T06:53:58.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:57.748+0000 7fe518635200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:53:58.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:57 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:57.841+0000 7fe518635200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:53:58.300 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/101455915' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-10T06:53:58.300 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-mon[25700]: mgrmap e9: y(active, since 19s) 2024-02-10T06:53:58.593 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:58.500+0000 7fe518635200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:53:58.593 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:58.592+0000 7fe518635200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:53:58.849 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:58.733+0000 7fe518635200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:53:58.849 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:58.831+0000 7fe518635200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:53:59.190 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:58 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:58.918+0000 7fe518635200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:53:59.191 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:59.190+0000 7fe518635200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:53:59.487 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2161656771' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T06:53:59.487 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:59.304+0000 7fe518635200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:53:59.488 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:59.392+0000 7fe518635200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:53:59.488 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:59.486+0000 7fe518635200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:53:59.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:53:59 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:53:59.665+0000 7fe518635200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:53:59.891 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:53:59.891 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for the mgr to restart... 2024-02-10T06:53:59.892 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr epoch 9... 2024-02-10T06:54:00.818 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:00 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:00.681+0000 7fe518635200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:54:00.818 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:00 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:00.817+0000 7fe518635200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:54:01.452 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:01 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:01.159+0000 7fe518635200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:54:01.453 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:01 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:01.253+0000 7fe518635200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:54:01.453 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:01 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:01.451+0000 7fe518635200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:54:01.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:01 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:01.634+0000 7fe518635200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:54:02.177 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:01 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:01.918+0000 7fe518635200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:54:02.177 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:02 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:02.176+0000 7fe518635200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:54:02.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:02 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:02.410+0000 7fe518635200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:54:03.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:03 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:03.206+0000 7fe518635200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:54:04.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:03 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:03.774+0000 7fe518635200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:54:04.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:03 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:03.873+0000 7fe518635200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:54:04.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:54:04.088+0000 7fe518635200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:54:04.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: Active manager daemon y restarted 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: Activating manager daemon y 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: osdmap e3: 0 total, 0 up, 0 in 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: mgrmap e10: y(active, starting, since 0.00319836s) 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: Manager daemon y is now available 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:04.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:05.335 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:54:05.335 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:54:05.335 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:05 smithi076 ceph-mon[25700]: mgrmap e11: y(active, since 1.00554s) 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:mgr epoch 9 is available 2024-02-10T06:54:06.068 INFO:teuthology.orchestra.run.smithi076.stdout:Generating a dashboard self-signed certificate... 2024-02-10T06:54:06.434 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: [10/Feb/2024:06:54:04] ENGINE Bus STARTING 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: [10/Feb/2024:06:54:04] ENGINE Serving on https://172.21.15.76:7150 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: [10/Feb/2024:06:54:05] ENGINE Serving on http://172.21.15.76:8765 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: [10/Feb/2024:06:54:05] ENGINE Bus STARTED 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T06:54:06.435 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:06 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:07.557 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-10T06:54:07.557 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial admin user... 2024-02-10T06:54:07.811 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:07 smithi076 ceph-mon[25700]: mgrmap e12: y(active, since 2s) 2024-02-10T06:54:07.811 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:07 smithi076 ceph-mon[25700]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:07.811 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:07.811 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:07.811 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:09.200 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$adWj/wn0SUmOrcY/0PaP3e/KnlpaKrvW4Yjc5Sq2/42D6XU7ruwv6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707548048, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-10T06:54:09.200 INFO:teuthology.orchestra.run.smithi076.stdout:Fetching dashboard port number... 2024-02-10T06:54:09.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:09 smithi076 ceph-mon[25700]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:09.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:10.780 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:10 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/4215295303' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-10T06:54:10.900 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 8443 2024-02-10T06:54:10.909 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:54:10.909 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-02-10T06:54:10.916 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:54:10.916 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-02-10T06:54:10.916 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-02-10T06:54:10.916 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout:Ceph Dashboard is now available at: 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout: URL: https://smithi076.front.sepia.ceph.com:8443/ 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout: User: admin 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout: Password: tmloj5g2ct 2024-02-10T06:54:10.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:10.919 INFO:teuthology.orchestra.run.smithi076.stdout:Saving cluster configuration to /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config directory 2024-02-10T06:54:10.919 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling autotune for osd_memory_target 2024-02-10T06:54:11.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:11 smithi076 ceph-mon[25700]: mgrmap e13: y(active, since 6s) 2024-02-10T06:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout:Or, if you are only running a single cluster on this host: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: ceph telemetry on 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout:For more information see: 2024-02-10T06:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-10T06:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout:Bootstrap complete. 2024-02-10T06:54:13.555 INFO:tasks.cephadm:Fetching config... 2024-02-10T06:54:13.556 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:54:13.556 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-10T06:54:13.571 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-10T06:54:13.571 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:54:13.571 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-10T06:54:13.626 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-10T06:54:13.626 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:54:13.626 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-10T06:54:13.693 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-10T06:54:13.693 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:54:13.693 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-10T06:54:13.748 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-10T06:54:13.748 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T06:54:13.828 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:54:13.863 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T06:54:13.909 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:54:13.924 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T06:54:13.969 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3RjI8YO2lKc3l3iRAjro2aen/0j6OSCgwNQCLWt8147eX4pqTPlKlJLIkVBRBEVhA/dfzBVAeXctcIvPKFYJiPT/6HMjebtGaOPAfia5orL6yWBQ87ZuikLUng9rA1IMqlLP8FXrfkAT/17rRxWiKk+h3BL+0o8nLSwfPl6i+a/JZdPbsr6G2x9yyzlAbCQl32SmRm9zJtysucpsOQAE8VSy9QB8tNkBL5B9ki+BTdFi+aNAaB4kYh8r1eQUjUDPxFmdExJqmDoKjCgmCmgrAIQFZ3nmzZoA+OMfyzOMZKpUr5MNEjuvDcwZUbUdwk/JXvIuYyTFZ4qTCOj1eD5OnOhnCUPEdgNLWwDkGobaL3HA82aL2Rmn23vlT9KK8DLywbCYq468Q6m022solVgOS1GEAaMnDAYDxrNHVzkaSnlM5viHLS1LuXNd3NqA72nt2LcDOSqCuf2y3a2YxBfH5toMZDVkCbBLQsELV3Zxn+9dsWnnH5Ws0K1nvV6wpwx0= ceph-e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:54:13.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-10T06:54:14.081 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:13 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/1480996580' entity='client.admin' 2024-02-10T06:54:14.227 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:54:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:16.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:16.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:17.635 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-10T06:54:17.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-10T06:54:17.891 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:18.262 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:17 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/1814272556' entity='client.admin' 2024-02-10T06:54:18.262 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:17 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:18.262 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:17 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:20.906 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-02-10T06:54:20.906 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:54:20.906 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-02-10T06:54:20.924 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:54:20.925 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:20.982 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-02-10T06:54:20.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch host add smithi136 2024-02-10T06:54:21.217 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:54:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:22.292 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:22 smithi076 ceph-mon[25700]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:54:22.292 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:22 smithi076 ceph-mon[25700]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:22.292 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:22 smithi076 ceph-mon[25700]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:54:23.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:23 smithi076 ceph-mon[25700]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:25 smithi076 ceph-mon[25700]: Deploying cephadm binary to smithi136 2024-02-10T06:54:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:25 smithi076 ceph-mon[25700]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:26 smithi076 ceph-mon[25700]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:27.437 INFO:teuthology.orchestra.run.smithi076.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-02-10T06:54:27.990 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T06:54:28.229 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:28.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:28 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:28.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:28 smithi076 ceph-mon[25700]: Added host smithi136 2024-02-10T06:54:28.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:28 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:28.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:28 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:28.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:28 smithi076 ceph-mon[25700]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:29.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:30.042 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:30.043 INFO:teuthology.orchestra.run.smithi076.stdout:[{"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-02-10T06:54:30.720 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-02-10T06:54:30.721 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:54:30.721 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-02-10T06:54:30.738 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:54:30.739 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:30.797 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-02-10T06:54:30.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch host add smithi189 2024-02-10T06:54:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:31.412 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:31 smithi076 ceph-mon[25700]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:54:31.413 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:31 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:31.413 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:31 smithi076 ceph-mon[25700]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:33.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:33 smithi076 ceph-mon[25700]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:34.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: Deploying cephadm binary to smithi189 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:35.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:35 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:36.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:36 smithi076 ceph-mon[25700]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:37.186 INFO:teuthology.orchestra.run.smithi076.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-02-10T06:54:37.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T06:54:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:38.484 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:38.484 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:38 smithi076 ceph-mon[25700]: Added host smithi189 2024-02-10T06:54:38.484 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:38.484 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:38.484 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:39.455 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:39 smithi076 ceph-mon[25700]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:39.892 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:54:39.893 INFO:teuthology.orchestra.run.smithi076.stdout:[{"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-02-10T06:54:40.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:40 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:40.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:40 smithi076 ceph-mon[25700]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:54:40.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:40 smithi076 ceph-mon[25700]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:40.611 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-10T06:54:40.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd crush tunables default 2024-02-10T06:54:40.846 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:54:43.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:43.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:43.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2833539391' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T06:54:43.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:43.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:43.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:43.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:54:43.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:43.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-02-10T06:54:44.180 INFO:tasks.cephadm:Adding mon.a on smithi076 2024-02-10T06:54:44.181 INFO:tasks.cephadm:Adding mon.b on smithi136 2024-02-10T06:54:44.181 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-02-10T06:54:44.181 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply mon '3;smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c' 2024-02-10T06:54:44.409 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2833539391' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T06:54:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:44 smithi076 ceph-mon[25700]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:54:45.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:45 smithi076 ceph-mon[25700]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:54:45.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:45 smithi076 ceph-mon[25700]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:46.118 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-02-10T06:54:46.690 DEBUG:teuthology.orchestra.run.smithi136:mon.b> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.b.service 2024-02-10T06:54:46.693 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.c.service 2024-02-10T06:54:46.695 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:54:46.695 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:54:47.337 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:54:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:54:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: Saving service mon spec with placement smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c;count:3 2024-02-10T06:54:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:47.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:47 smithi076 ceph-mon[25700]: Deploying daemon mon.c on smithi189 2024-02-10T06:54:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:49 smithi076 ceph-mon[25700]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:54:50.747 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","modified":"2024-02-10T06:53:07.659968Z","created":"2024-02-10T06:53:07.659968Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-10T06:54:50.749 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-02-10T06:54:52.426 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:54:52.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:54:52.672 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:54:56.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: mon.a calling monitor election 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: mon.c calling monitor election 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: monmap e2: 2 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: fsmap 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: mgrmap e13: y(active, since 52s) 2024-02-10T06:54:56.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: overall HEALTH_OK 2024-02-10T06:54:56.537 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:56.537 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:56.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:56 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:57.436 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:57.436 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:57.436 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:54:57.437 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:57.437 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: Deploying daemon mon.b on smithi136 2024-02-10T06:54:57.437 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:57 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:57.437 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:54:57.437 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","modified":"2024-02-10T06:54:51.079786Z","created":"2024-02-10T06:53:07.659968Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T06:54:57.439 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: Deploying daemon mon.b on smithi136 2024-02-10T06:54:57.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:57 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:54:58.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:58 smithi189 ceph-mon[30417]: from='client.? 172.21.15.189:0/2467224921' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:54:58.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:58 smithi076 ceph-mon[25700]: from='client.? 172.21.15.189:0/2467224921' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:54:59.029 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:54:59.029 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:54:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:54:59.394 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:54:59 smithi189 ceph-mon[30417]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:59.449 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:54:59 smithi076 ceph-mon[25700]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:54:59.732 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 podman[29705]: 2024-02-10 06:54:59.53467214 +0000 UTC m=+0.018080540 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 podman[29705]: 2024-02-10 06:54:59.800232076 +0000 UTC m=+0.283640479 container init 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 podman[29705]: 2024-02-10 06:54:59.805681269 +0000 UTC m=+0.289089670 container start 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-290212a, CEPH_POINT_RELEASE=) 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pidfile_write: ignore empty --pid-file 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: load: jerasure load: lrc 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: RocksDB version: 7.9.2 2024-02-10T06:55:00.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Git sha 0 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: DB SUMMARY 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: DB Session ID: GAXGCT0QHP9OZJGTV777 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: CURRENT file: CURRENT 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.error_if_exists: 0 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.create_if_missing: 0 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.paranoid_checks: 1 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T06:55:00.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T06:55:00.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.env: 0x56363e97bae0 2024-02-10T06:55:00.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T06:55:00.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.info_log: 0x56363f54b8c0 2024-02-10T06:55:00.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T06:55:00.139 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.statistics: (nil) 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.use_fsync: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_log_file_size: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_fallocate: 1 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.use_direct_reads: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T06:55:00.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.db_log_dir: 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.wal_dir: 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.write_buffer_manager: 0x56363f51f400 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T06:55:00.141 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.unordered_write: 0 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.row_cache: None 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.wal_filter: None 2024-02-10T06:55:00.142 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.two_write_queues: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.wal_compression: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.atomic_flush: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.log_readahead_size: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T06:55:00.143 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_background_jobs: 2 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_background_compactions: -1 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_subcompactions: 1 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_open_files: -1 2024-02-10T06:55:00.144 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_background_flushes: -1 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Compression algorithms supported: 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kZSTD supported: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kXpressCompression supported: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kZlibCompression supported: 1 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kSnappyCompression supported: 1 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kLZ4Compression supported: 1 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: kBZip2Compression supported: 0 2024-02-10T06:55:00.145 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.merge_operator: 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_filter: None 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56363f54b9c0) 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout: cache_index_and_filter_blocks: 1 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T06:55:00.146 INFO:journalctl@ceph.mon.b.smithi136.stdout: pin_top_level_index_and_filter: 1 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_type: 0 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: data_block_index_type: 0 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_shortening: 1 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: checksum: 4 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: no_block_cache: 0 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache: 0x56363f54f090 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_name: BinnedLRUCache 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_options: 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: capacity : 536870912 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: num_shard_bits : 4 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: strict_capacity_limit : 0 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: high_pri_pool_ratio: 0.000 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_cache_compressed: (nil) 2024-02-10T06:55:00.147 INFO:journalctl@ceph.mon.b.smithi136.stdout: persistent_cache: (nil) 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_size: 4096 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_size_deviation: 10 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_restart_interval: 16 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: index_block_restart_interval: 1 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: metadata_block_size: 4096 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: partition_filters: 0 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: use_delta_encoding: 1 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: filter_policy: bloomfilter 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: whole_key_filtering: 1 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: verify_compression: 0 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: read_amp_bytes_per_bit: 0 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: format_version: 5 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: enable_index_compression: 1 2024-02-10T06:55:00.148 INFO:journalctl@ceph.mon.b.smithi136.stdout: block_align: 0 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout: max_auto_readahead_size: 262144 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout: prepopulate_block_cache: 0 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout: initial_auto_readahead_size: 8192 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression: NoCompression 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.num_levels: 7 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T06:55:00.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T06:55:00.150 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T06:55:00.151 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T06:55:00.152 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.table_properties_collectors: 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.inplace_update_support: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.bloom_locality: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.max_successive_merges: 0 2024-02-10T06:55:00.153 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.ttl: 2592000 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enable_blob_files: false 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.min_blob_size: 0 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T06:55:00.154 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed783628-5557-448b-ba72-2d8072fb8534 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707548099835447, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707548099835976, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707548099, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed783628-5557-448b-ba72-2d8072fb8534", "db_session_id": "GAXGCT0QHP9OZJGTV777", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707548099836078, "job": 1, "event": "recovery_finished"} 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56363f640000 2024-02-10T06:55:00.155 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: DB pointer 0x56363f62a000 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** DB Stats ** 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** Compaction Stats [default] ** 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T06:55:00.156 INFO:journalctl@ceph.mon.b.smithi136.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** Compaction Stats [default] ** 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T06:55:00.157 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: Block cache BinnedLRUCache@0x56363f54f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: using public_addr v2:172.21.15.136:0/0 -> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: starting mon.b rank -1 at public addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] at bind addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(???) e0 preinit fsid e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).mds e1 new map 2024-02-10T06:55:00.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: e1 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: legacy client fscid: -1 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout: No filesystems configured 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T06:55:00.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:55:00.160 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Deploying cephadm binary to smithi189 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Added host smithi189 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.161 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/2833539391' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.162 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/2833539391' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Saving service mon spec with placement smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c;count:3 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.163 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Deploying daemon mon.c on smithi189 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.a calling monitor election 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.c calling monitor election 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.164 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: monmap e2: 2 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: fsmap 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mgrmap e13: y(active, since 52s) 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: overall HEALTH_OK 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: Deploying daemon mon.b on smithi136 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:00.165 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: from='client.? 172.21.15.189:0/2467224921' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:55:00.166 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:00.166 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 ceph-mon[29743]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T06:55:00.166 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 bash[29705]: 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 2024-02-10T06:55:00.166 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:54:59 smithi136 systemd[1]: Started Ceph mon.b for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:55:05.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: mon.a calling monitor election 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: mon.c calling monitor election 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: fsmap 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: mgrmap e13: y(active, since 60s) 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: mon.b (rank 2) addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] is down (out of quorum) 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.287 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:55:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: mon.a calling monitor election 2024-02-10T06:55:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: mon.c calling monitor election 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:05.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: fsmap 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: mgrmap e13: y(active, since 60s) 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: mon.b (rank 2) addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] is down (out of quorum) 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.423 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:05.523 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-10T06:55:05.523 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","modified":"2024-02-10T06:54:59.863283Z","created":"2024-02-10T06:53:07.659968Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T06:55:05.525 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-02-10T06:55:06.028 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-10T06:55:06.029 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph config generate-minimal-conf 2024-02-10T06:55:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:55:06.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:06.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:05 smithi076 ceph-mon[25700]: from='client.? 172.21.15.189:0/801643083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:55:06.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:06.387 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:06.387 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:05 smithi189 ceph-mon[30417]: from='client.? 172.21.15.189:0/801643083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:55:06.387 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: mon.a calling monitor election 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: mon.c calling monitor election 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: fsmap 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: mgrmap e13: y(active, since 60s) 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T06:55:07.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: mon.b (rank 2) addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] is down (out of quorum) 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='client.? 172.21.15.189:0/801643083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:55:07.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:08.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mon.b calling monitor election 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mon.b calling monitor election 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mon.c calling monitor election 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mon.a calling monitor election 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: fsmap 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: mgrmap e13: y(active, since 62s) 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T06:55:08.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Cluster is now healthy 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: overall HEALTH_OK 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.032 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:08.032 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:08.032 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mon.b calling monitor election 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mon.b calling monitor election 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mon.c calling monitor election 2024-02-10T06:55:08.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mon.a calling monitor election 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: fsmap 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: mgrmap e13: y(active, since 62s) 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Cluster is now healthy 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: overall HEALTH_OK 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:08.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:08.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mon.b calling monitor election 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mon.b calling monitor election 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mon.c calling monitor election 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mon.a calling monitor election 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: monmap e3: 3 mons at {a=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],b=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: fsmap 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: mgrmap e13: y(active, since 62s) 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T06:55:08.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Cluster is now healthy 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: overall HEALTH_OK 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:08.265 INFO:teuthology.orchestra.run.smithi076.stdout:# minimal ceph.conf for e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:55:08.265 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-02-10T06:55:08.266 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:55:08.266 INFO:teuthology.orchestra.run.smithi076.stdout: mon_host = [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-02-10T06:55:08.985 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:08 smithi076 ceph-mon[25700]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:55:08.986 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:08 smithi076 ceph-mon[25700]: Reconfiguring daemon mon.a on smithi076 2024-02-10T06:55:08.986 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:08 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:08.986 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:08 smithi076 ceph-mon[25700]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:08.986 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:08 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/1055587105' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:09.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:08 smithi136 ceph-mon[29743]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:55:09.139 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:08 smithi136 ceph-mon[29743]: Reconfiguring daemon mon.a on smithi076 2024-02-10T06:55:09.139 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:08 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:09.139 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:08 smithi136 ceph-mon[29743]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:09.139 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:08 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/1055587105' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:09.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:08 smithi189 ceph-mon[30417]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:55:09.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:08 smithi189 ceph-mon[30417]: Reconfiguring daemon mon.a on smithi076 2024-02-10T06:55:09.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:08 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:55:09.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:08 smithi189 ceph-mon[30417]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:09.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:08 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/1055587105' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:09.773 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-10T06:55:09.773 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:55:09.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T06:55:09.803 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:55:09.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:55:09.871 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:55:09.871 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T06:55:09.903 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:55:09.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:55:09.971 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:55:09.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T06:55:10.006 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:55:10.006 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:55:10.078 INFO:tasks.cephadm:Adding mgr.y on smithi076 2024-02-10T06:55:10.078 INFO:tasks.cephadm:Adding mgr.x on smithi136 2024-02-10T06:55:10.079 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-02-10T06:55:10.079 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply mgr '3;smithi076=y;smithi136=x;smithi189=z' 2024-02-10T06:55:10.353 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:10.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:10.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: Reconfiguring daemon mon.b on smithi136 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:11 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: Reconfiguring daemon mon.b on smithi136 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:11.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:11.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:11 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: Reconfiguring daemon mon.b on smithi136 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:55:11.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:11 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:12.360 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:12 smithi189 ceph-mon[30417]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:55:12.360 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:12 smithi189 ceph-mon[30417]: Reconfiguring daemon mon.c on smithi189 2024-02-10T06:55:12.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:12 smithi076 ceph-mon[25700]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:55:12.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:12 smithi076 ceph-mon[25700]: Reconfiguring daemon mon.c on smithi189 2024-02-10T06:55:12.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:12 smithi136 ceph-mon[29743]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:55:12.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:12 smithi136 ceph-mon[29743]: Reconfiguring daemon mon.c on smithi189 2024-02-10T06:55:12.691 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-02-10T06:55:13.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:13 smithi189 ceph-mon[30417]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:13.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:13 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:13 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:13 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:13 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:13.433 DEBUG:teuthology.orchestra.run.smithi136:mgr.x> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x.service 2024-02-10T06:55:13.437 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.z.service 2024-02-10T06:55:13.440 INFO:tasks.cephadm:Deploying OSDs... 2024-02-10T06:55:13.440 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:55:13.440 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:55:13.456 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:55:13.456 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/nvme1n1 2024-02-10T06:55:13.462 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:13 smithi136 ceph-mon[29743]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:13.463 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:13 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.463 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:13 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.463 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:13 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.463 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:13 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/nvme1n1 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:54:16.891332748 +0000 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:52:03.477572086 +0000 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:52:03.477572086 +0000 2024-02-10T06:55:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:52:03.464572492 +0000 2024-02-10T06:55:13.511 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T06:55:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:13 smithi076 ceph-mon[25700]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:13 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:13 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:13 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:13 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:13.561 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:55:13.561 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:55:13.561 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000282639 s, 1.8 MB/s 2024-02-10T06:55:13.562 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T06:55:13.618 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/nvme2n1 2024-02-10T06:55:13.674 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/nvme2n1 2024-02-10T06:55:13.674 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-10T06:55:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:13.675 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:54:16.901332435 +0000 2024-02-10T06:55:13.675 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:52:03.621567581 +0000 2024-02-10T06:55:13.675 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:52:03.621567581 +0000 2024-02-10T06:55:13.675 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:52:03.610567925 +0000 2024-02-10T06:55:13.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T06:55:13.739 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:55:13.740 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:55:13.740 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000214438 s, 2.4 MB/s 2024-02-10T06:55:13.741 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T06:55:13.797 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/nvme3n1 2024-02-10T06:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/nvme3n1 2024-02-10T06:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:54:16.912332091 +0000 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:52:03.770562919 +0000 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:52:03.770562919 +0000 2024-02-10T06:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:52:03.758563295 +0000 2024-02-10T06:55:13.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T06:55:13.918 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:55:13.918 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:55:13.918 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000213863 s, 2.4 MB/s 2024-02-10T06:55:13.919 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T06:55:13.975 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/nvme4n1 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/nvme4n1 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-10 06:54:16.923331747 +0000 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-10 06:52:03.935557758 +0000 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-10 06:52:03.935557758 +0000 2024-02-10T06:55:14.030 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-10 06:52:03.926558039 +0000 2024-02-10T06:55:14.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T06:55:14.096 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-10T06:55:14.096 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-10T06:55:14.096 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000240289 s, 2.1 MB/s 2024-02-10T06:55:14.097 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T06:55:14.153 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:55:14.153 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:55:14.171 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:55:14.172 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/nvme1n1 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/nvme1n1 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 763 Links: 1 Device type: 103,2 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:54:32.326024601 +0000 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:52:05.095334352 +0000 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:52:05.095334352 +0000 2024-02-10T06:55:14.230 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:52:05.087334111 +0000 2024-02-10T06:55:14.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T06:55:14.300 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:55:14.300 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:55:14.300 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000260818 s, 2.0 MB/s 2024-02-10T06:55:14.301 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T06:55:14.359 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/nvme2n1 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/nvme2n1 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:54:32.338024331 +0000 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:52:05.240338723 +0000 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:52:05.240338723 +0000 2024-02-10T06:55:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:52:05.235338572 +0000 2024-02-10T06:55:14.417 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T06:55:14.482 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=y;smithi136=x;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: Saving service mgr spec with placement smithi076=y;smithi136=x;smithi189=z;count:3 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:14.483 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:14 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=y;smithi136=x;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: Saving service mgr spec with placement smithi076=y;smithi136=x;smithi189=z;count:3 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:14.484 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:14 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.485 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:55:14.485 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:55:14.485 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000227712 s, 2.2 MB/s 2024-02-10T06:55:14.486 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=y;smithi136=x;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: Saving service mgr spec with placement smithi076=y;smithi136=x;smithi189=z;count:3 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:14.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:14 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:14.543 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/nvme3n1 2024-02-10T06:55:14.599 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/nvme3n1 2024-02-10T06:55:14.599 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:54:32.349024084 +0000 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:52:05.396343426 +0000 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:52:05.396343426 +0000 2024-02-10T06:55:14.600 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:52:05.384343064 +0000 2024-02-10T06:55:14.600 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T06:55:14.666 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:55:14.666 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:55:14.667 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000214374 s, 2.4 MB/s 2024-02-10T06:55:14.668 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T06:55:14.723 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/nvme4n1 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/nvme4n1 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-02-10 06:54:32.360023836 +0000 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-02-10 06:52:05.544345334 +0000 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-02-10 06:52:05.544345334 +0000 2024-02-10T06:55:14.779 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-02-10 06:52:05.532345605 +0000 2024-02-10T06:55:14.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T06:55:14.845 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-02-10T06:55:14.845 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-02-10T06:55:14.845 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000211288 s, 2.4 MB/s 2024-02-10T06:55:14.846 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T06:55:14.902 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:55:14.902 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:55:14.919 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T06:55:14.919 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme1n1 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme1n1 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 763 Links: 1 Device type: 103,2 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:54:41.491248693 +0000 2024-02-10T06:55:14.974 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:52:06.756203146 +0000 2024-02-10T06:55:14.975 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:52:06.756203146 +0000 2024-02-10T06:55:14.975 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:52:06.748203298 +0000 2024-02-10T06:55:14.975 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T06:55:15.041 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:55:15.041 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:55:15.041 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000230448 s, 2.2 MB/s 2024-02-10T06:55:15.042 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T06:55:15.099 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme2n1 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme2n1 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:54:41.502248484 +0000 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:52:06.919200035 +0000 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:52:06.919200035 +0000 2024-02-10T06:55:15.154 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:52:06.910200206 +0000 2024-02-10T06:55:15.154 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T06:55:15.203 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:15 smithi189 ceph-mon[30417]: Deploying daemon mgr.z on smithi189 2024-02-10T06:55:15.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:15 smithi189 ceph-mon[30417]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:15.220 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:55:15.220 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:55:15.220 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00024906 s, 2.1 MB/s 2024-02-10T06:55:15.221 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T06:55:15.277 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme3n1 2024-02-10T06:55:15.338 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme3n1 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:54:41.513248275 +0000 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:52:07.024198031 +0000 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:52:07.024198031 +0000 2024-02-10T06:55:15.339 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:52:07.018198145 +0000 2024-02-10T06:55:15.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T06:55:15.453 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:55:15.453 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:55:15.453 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000218 s, 2.3 MB/s 2024-02-10T06:55:15.454 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T06:55:15.470 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme4n1 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme4n1 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-10 06:54:41.524248065 +0000 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-10 06:52:07.179195072 +0000 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-10 06:52:07.179195072 +0000 2024-02-10T06:55:15.532 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-10 06:52:07.168195282 +0000 2024-02-10T06:55:15.533 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T06:55:15.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:15 smithi076 ceph-mon[25700]: Deploying daemon mgr.z on smithi189 2024-02-10T06:55:15.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:15 smithi076 ceph-mon[25700]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:15.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:15 smithi136 ceph-mon[29743]: Deploying daemon mgr.z on smithi189 2024-02-10T06:55:15.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:15 smithi136 ceph-mon[29743]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:15.641 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-10T06:55:15.641 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-10T06:55:15.642 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00023398 s, 2.2 MB/s 2024-02-10T06:55:15.643 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T06:55:15.658 INFO:tasks.cephadm:Deploying osd.0 on smithi076 with /dev/nvme4n1... 2024-02-10T06:55:15.658 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T06:55:15.888 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:15 smithi189 systemd[1]: Starting Ceph mgr.z for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:55:15.899 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:55:16.142 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:15 smithi189 podman[32470]: 2024-02-10 06:55:15.790676843 +0000 UTC m=+0.017821234 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:55:16.143 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:15 smithi189 podman[32470]: 2024-02-10 06:55:15.909463962 +0000 UTC m=+0.136608338 container create 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-10T06:55:16.143 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 podman[32470]: 2024-02-10 06:55:16.142424342 +0000 UTC m=+0.369568719 container init 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-10T06:55:16.405 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 podman[32470]: 2024-02-10 06:55:16.148035402 +0000 UTC m=+0.375179786 container start 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, RELEASE=main-290212a) 2024-02-10T06:55:16.405 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mgr[32508]: -- 172.21.15.189:0/869883827 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b183861a0 con 0x564b1835d000 2024-02-10T06:55:16.405 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 bash[32470]: 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e 2024-02-10T06:55:16.406 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 systemd[1]: Started Ceph mgr.z for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:16.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:16.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:16 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:16 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:17.171 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:16.723+0000 7f8a5ca7f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:55:17.171 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:16 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:16.818+0000 7f8a5ca7f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:55:17.470 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-mon[30417]: Deploying daemon mgr.x on smithi136 2024-02-10T06:55:17.722 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:17 smithi136 ceph-mon[29743]: Deploying daemon mgr.x on smithi136 2024-02-10T06:55:17.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:17 smithi076 ceph-mon[25700]: Deploying daemon mgr.x on smithi136 2024-02-10T06:55:17.798 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:17.469+0000 7f8a5ca7f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:55:17.798 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:17.561+0000 7f8a5ca7f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:55:17.798 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:17.702+0000 7f8a5ca7f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123044 s, 852 MB/s 2024-02-10T06:55:17.871 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-02-10T06:55:18.146 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:17.797+0000 7f8a5ca7f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:55:18.146 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:17 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:17.880+0000 7f8a5ca7f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:55:18.146 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:18.145+0000 7f8a5ca7f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:55:18.411 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:18.255+0000 7f8a5ca7f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:55:18.411 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:18.341+0000 7f8a5ca7f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:55:18.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:18 smithi076 ceph-mon[25700]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:18.442 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi076:/dev/nvme4n1 2024-02-10T06:55:18.633 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:18 smithi136 ceph-mon[29743]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:18.634 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 systemd[1]: Starting Ceph mgr.x for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:55:18.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-mon[30417]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:18.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:18.437+0000 7f8a5ca7f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:55:18.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:18 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:18.614+0000 7f8a5ca7f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:55:18.675 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:55:18.886 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 podman[31175]: 2024-02-10 06:55:18.633549578 +0000 UTC m=+0.106937136 container create 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-10T06:55:18.886 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 podman[31175]: 2024-02-10 06:55:18.5443811 +0000 UTC m=+0.017768672 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:55:18.886 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 podman[31175]: 2024-02-10 06:55:18.844180763 +0000 UTC m=+0.317568325 container init 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:55:18.886 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 podman[31175]: 2024-02-10 06:55:18.849514371 +0000 UTC m=+0.322901928 container start 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-10T06:55:18.886 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 ceph-mgr[31214]: -- 172.21.15.136:0/703006540 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dac6461a0 con 0x560dac61c800 2024-02-10T06:55:19.266 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 bash[31175]: 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 2024-02-10T06:55:19.266 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:18 smithi136 systemd[1]: Started Ceph mgr.x for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:55:19.522 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:19 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:19.420+0000 7f4ab9aaa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:55:19.522 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:19 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:19.520+0000 7f4ab9aaa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:55:19.921 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:19 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:19.614+0000 7f8a5ca7f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:55:19.921 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:19 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:19.760+0000 7f8a5ca7f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:55:20.121 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.121 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.121 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.122 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.122 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.122 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:20.314 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.219+0000 7f4ab9aaa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:55:20.314 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.312+0000 7f4ab9aaa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:55:20.314 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.314 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.314 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.315 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.315 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.315 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:20.399 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:20.107+0000 7f8a5ca7f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:55:20.399 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:20.203+0000 7f8a5ca7f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:55:20.399 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:20.398+0000 7f8a5ca7f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:20.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:20.565 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.460+0000 7f4ab9aaa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:55:20.565 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.563+0000 7f4ab9aaa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:55:20.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:20.593+0000 7f8a5ca7f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:55:20.885 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.649+0000 7f4ab9aaa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:20 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:20.878+0000 7f8a5ca7f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:21.139+0000 7f8a5ca7f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:21.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:21.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:21.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.198 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:20 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:20.913+0000 7f4ab9aaa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:21.019+0000 7f4ab9aaa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:21.105+0000 7f4ab9aaa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:21.196+0000 7f4ab9aaa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.199 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:21.200 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.200 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:21.200 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:21.200 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:21.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:55:21.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:55:21.286 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:21 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:21.421 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:21 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:21.373+0000 7f8a5ca7f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:55:21.636 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:21 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:21.373+0000 7f4ab9aaa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:55:22.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:22 smithi076 ceph-mon[25700]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:22.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:22 smithi076 ceph-mon[25700]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:55:22.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:22 smithi076 ceph-mon[25700]: Reconfiguring daemon mgr.y on smithi076 2024-02-10T06:55:22.335 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-mon[29743]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:22.336 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-mon[29743]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:55:22.336 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-mon[29743]: Reconfiguring daemon mgr.y on smithi076 2024-02-10T06:55:22.421 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:22.170+0000 7f8a5ca7f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:55:22.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-mon[30417]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:22.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-mon[30417]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:55:22.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-mon[30417]: Reconfiguring daemon mgr.y on smithi076 2024-02-10T06:55:22.635 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:22.335+0000 7f4ab9aaa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:55:22.636 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:22.464+0000 7f4ab9aaa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:55:23.033 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:22.742+0000 7f8a5ca7f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:55:23.033 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:22 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:22.837+0000 7f8a5ca7f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:55:23.090 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:23 smithi136 ceph-mon[29743]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:23.090 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:22.806+0000 7f4ab9aaa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:55:23.090 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:22 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:22.896+0000 7f4ab9aaa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:55:23.091 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:23 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:23.089+0000 7f4ab9aaa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:55:23.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:23 smithi076 ceph-mon[25700]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:23.385 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:23 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:23.271+0000 7f4ab9aaa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:55:23.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:23 smithi189 ceph-mon[30417]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:23.421 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:55:23 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:55:23.050+0000 7f8a5ca7f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:55:23.843 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:23 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:23.556+0000 7f4ab9aaa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:55:23.844 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:23 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:23.843+0000 7f4ab9aaa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:55:24.135 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:24.079+0000 7f4ab9aaa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: Standby manager daemon z started 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:24.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:24.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: Standby manager daemon z started 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:24.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:24.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:24.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:24 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: Standby manager daemon z started 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/3376135590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:24.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:24.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:24.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:24.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:24 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:24 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:24.876+0000 7f4ab9aaa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]: dispatch 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]': finished 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:55:25.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]: dispatch 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]': finished 2024-02-10T06:55:25.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:55:25.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:25 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T06:55:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:55:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]: dispatch 2024-02-10T06:55:25.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/290588716' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39fc88c9-e027-42a1-8518-66357f266a96"}]': finished 2024-02-10T06:55:25.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:55:25.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:25 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:25.757 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:25.449+0000 7f4ab9aaa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:55:25.758 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:25.546+0000 7f4ab9aaa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:55:25.758 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:55:25 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:55:25.757+0000 7f4ab9aaa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/445701451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: Standby manager daemon x started 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:26 smithi189 ceph-mon[30417]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/445701451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: Standby manager daemon x started 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:26.814 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:26 smithi076 ceph-mon[25700]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/445701451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: Standby manager daemon x started 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/1349228731' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:55:26.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:26 smithi136 ceph-mon[29743]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:27.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:27 smithi189 ceph-mon[30417]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-10T06:55:27.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:27 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:55:27.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:27 smithi076 ceph-mon[25700]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-10T06:55:27.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:27 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:55:27.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:27 smithi136 ceph-mon[29743]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-10T06:55:27.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:27 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:55:28.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:28 smithi189 ceph-mon[30417]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:28.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:28 smithi076 ceph-mon[25700]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:28.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:28 smithi136 ceph-mon[29743]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:31.316 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:31 smithi076 ceph-mon[25700]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:31.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:31 smithi189 ceph-mon[30417]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:31.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:31 smithi136 ceph-mon[29743]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:32.228 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:32 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:55:32.228 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:32 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:32.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:32 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:55:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:32 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:32.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:32 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:55:32.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:32 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:33.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:33 smithi076 ceph-mon[25700]: Deploying daemon osd.0 on smithi076 2024-02-10T06:55:33.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:33 smithi076 ceph-mon[25700]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:33 smithi136 ceph-mon[29743]: Deploying daemon osd.0 on smithi076 2024-02-10T06:55:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:33 smithi136 ceph-mon[29743]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:33.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:33 smithi189 ceph-mon[30417]: Deploying daemon osd.0 on smithi076 2024-02-10T06:55:33.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:33 smithi189 ceph-mon[30417]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:35.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:35 smithi076 ceph-mon[25700]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:35.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:35 smithi136 ceph-mon[29743]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:35.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:35 smithi189 ceph-mon[30417]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:36.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:36 smithi189 ceph-mon[30417]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:36.758 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:36 smithi076 ceph-mon[25700]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:36.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:36 smithi136 ceph-mon[29743]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:38.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:38.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:38 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:38 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:38 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:38.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:38 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:38 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:38.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:38 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:39.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:39 smithi076 ceph-mon[25700]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:39.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:39 smithi136 ceph-mon[29743]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:39.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:39 smithi189 ceph-mon[30417]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:41.055 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:41 smithi076 ceph-mon[25700]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:55:41.096 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 0 on host 'smithi076' 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:41 smithi136 ceph-mon[29743]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:41 smithi189 ceph-mon[30417]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:55:41.708 DEBUG:teuthology.orchestra.run.smithi076:osd.0> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.0.service 2024-02-10T06:55:41.711 INFO:tasks.cephadm:Deploying osd.1 on smithi136 with /dev/nvme4n1... 2024-02-10T06:55:41.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T06:55:41.956 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.251 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.281 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:42 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:42 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:42.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0[44758]: 2024-02-10T06:55:42.483+0000 7f69d0c5b700 -1 osd.0 0 waiting for initial osdmap 2024-02-10T06:55:42.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:55:42 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0[44758]: 2024-02-10T06:55:42.486+0000 7f69c8a46700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:55:43.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-10T06:55:43.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:55:43.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:43.167 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:43 smithi076 ceph-mon[25700]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:43.178 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:43 smithi136 ceph-mon[29743]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:55:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:43 smithi189 ceph-mon[30417]: from='osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727]' entity='osd.0' 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0099355 s, 1.1 GB/s 2024-02-10T06:55:43.697 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: purged_snaps scrub starts 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: purged_snaps scrub ok 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727] boot 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:55:44.292 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.319 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi136:/dev/nvme4n1 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: purged_snaps scrub starts 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: purged_snaps scrub ok 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727] boot 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:55:44.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: purged_snaps scrub starts 2024-02-10T06:55:44.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: purged_snaps scrub ok 2024-02-10T06:55:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: osd.0 [v2:172.21.15.76:6802/1563965727,v1:172.21.15.76:6803/1563965727] boot 2024-02-10T06:55:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:55:44.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:55:44.553 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.347 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:45.348 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:45.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:55:46.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:46 smithi136 ceph-mon[29743]: Detected new or changed devices on smithi076 2024-02-10T06:55:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:46 smithi136 ceph-mon[29743]: Adjusting osd_memory_target on smithi076 to 17088M 2024-02-10T06:55:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:46 smithi136 ceph-mon[29743]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:55:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:46 smithi189 ceph-mon[30417]: Detected new or changed devices on smithi076 2024-02-10T06:55:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:46 smithi189 ceph-mon[30417]: Adjusting osd_memory_target on smithi076 to 17088M 2024-02-10T06:55:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:46 smithi189 ceph-mon[30417]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:55:46.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:46 smithi076 ceph-mon[25700]: Detected new or changed devices on smithi076 2024-02-10T06:55:46.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:46 smithi076 ceph-mon[25700]: Adjusting osd_memory_target on smithi076 to 17088M 2024-02-10T06:55:46.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:46 smithi076 ceph-mon[25700]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:55:47.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:47 smithi136 ceph-mon[29743]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:47.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:47 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:47.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:47 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:47.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:47 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:47.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:47 smithi189 ceph-mon[30417]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:47.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:47 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:47.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:47 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:47.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:47 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:47 smithi076 ceph-mon[25700]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:55:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:55:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:47 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:48.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:48 smithi136 ceph-mon[29743]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:48.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:48 smithi189 ceph-mon[30417]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:48.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:48 smithi076 ceph-mon[25700]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: from='client.? 172.21.15.136:0/2084457851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]': finished 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:55:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:49 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: from='client.? 172.21.15.136:0/2084457851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]': finished 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:55:49.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:49 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: from='client.? 172.21.15.136:0/2084457851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]: dispatch 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2e47e83-c869-4489-90cb-91199a0901bb"}]': finished 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:55:49.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:49 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:55:51.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:51 smithi076 ceph-mon[25700]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:51.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:51 smithi076 ceph-mon[25700]: from='client.? 172.21.15.136:0/1444471313' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:51.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:51 smithi136 ceph-mon[29743]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:51.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:51 smithi136 ceph-mon[29743]: from='client.? 172.21.15.136:0/1444471313' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:51.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:51 smithi189 ceph-mon[30417]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:51.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:51 smithi189 ceph-mon[30417]: from='client.? 172.21.15.136:0/1444471313' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:55:53.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:53 smithi136 ceph-mon[29743]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:53.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:53 smithi076 ceph-mon[25700]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:53.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:53 smithi189 ceph-mon[30417]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:55.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:55 smithi136 ceph-mon[29743]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:55.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:55 smithi189 ceph-mon[30417]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:55.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:55 smithi076 ceph-mon[25700]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:56.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:56 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:55:56.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:56 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:56.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:56 smithi189 ceph-mon[30417]: Deploying daemon osd.1 on smithi136 2024-02-10T06:55:56.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:56 smithi189 ceph-mon[30417]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:56.733 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:56 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:55:56.733 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:56 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:56.733 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:56 smithi136 ceph-mon[29743]: Deploying daemon osd.1 on smithi136 2024-02-10T06:55:56.733 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:56 smithi136 ceph-mon[29743]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:56.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:55:56.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:56 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:56.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:56 smithi076 ceph-mon[25700]: Deploying daemon osd.1 on smithi136 2024-02-10T06:55:56.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:56 smithi076 ceph-mon[25700]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:59.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:55:59 smithi136 ceph-mon[29743]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:59.487 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:55:59 smithi076 ceph-mon[25700]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:55:59.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:55:59 smithi189 ceph-mon[30417]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:01.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:01 smithi076 ceph-mon[25700]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:01.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:01 smithi136 ceph-mon[29743]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:01.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:01 smithi189 ceph-mon[30417]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:02.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:02 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:02 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:02 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:02.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:02 smithi136 ceph-mon[29743]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:02.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:02 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:02 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:02 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:02.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:02 smithi189 ceph-mon[30417]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:02.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:02 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:02 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:02.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:02 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:02.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:02 smithi076 ceph-mon[25700]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:04.810 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 1 on host 'smithi136' 2024-02-10T06:56:05.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:05.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:05.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.353 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:05.377 DEBUG:teuthology.orchestra.run.smithi136:osd.1> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.1.service 2024-02-10T06:56:05.379 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/nvme4n1... 2024-02-10T06:56:05.380 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:05.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:05.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:05.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:05.625 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:06.338 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-mon[29743]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:56:06.338 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-mon[29743]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:56:06.338 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:06.339 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-mon[29743]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.339 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-mon[29743]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:06 smithi189 ceph-mon[30417]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:56:06.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:06 smithi189 ceph-mon[30417]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:56:06.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:06 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:06.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:06 smithi189 ceph-mon[30417]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:06 smithi189 ceph-mon[30417]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:06 smithi076 ceph-mon[25700]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:56:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:06 smithi076 ceph-mon[25700]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:56:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:06 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:06 smithi076 ceph-mon[25700]: from='osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:06 smithi076 ceph-mon[25700]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-02-10T06:56:06.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1[34289]: 2024-02-10T06:56:06.488+0000 7f9e63f94700 -1 osd.1 0 waiting for initial osdmap 2024-02-10T06:56:06.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1[34289]: 2024-02-10T06:56:06.491+0000 7f9e5bd7f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:56:07.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:07.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-02-10T06:56:07.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:56:07.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:07 smithi189 ceph-mon[30417]: from='osd.1 ' entity='osd.1' 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:07 smithi076 ceph-mon[25700]: from='osd.1 ' entity='osd.1' 2024-02-10T06:56:07.537 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T06:56:07.537 INFO:teuthology.orchestra.run.smithi189.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T06:56:07.537 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T06:56:07.538 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-02-10T06:56:07.538 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-02-10T06:56:07.538 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130797 s, 802 MB/s 2024-02-10T06:56:07.538 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:07.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:07 smithi136 ceph-mon[29743]: from='osd.1 ' entity='osd.1' 2024-02-10T06:56:08.147 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi189:/dev/nvme4n1 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: purged_snaps scrub starts 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: purged_snaps scrub ok 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643] boot 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:56:08.371 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:08 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: purged_snaps scrub starts 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: purged_snaps scrub ok 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643] boot 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:56:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:08 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:08.422 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: purged_snaps scrub starts 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: purged_snaps scrub ok 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: osd.1 [v2:172.21.15.136:6800/3046476643,v1:172.21.15.136:6801/3046476643] boot 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:56:08.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:08 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:09.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:09 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:09.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:09 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:09.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:09 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: Detected new or changed devices on smithi136 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: Adjusting osd_memory_target on smithi136 to 17088M 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:56:10.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:10 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:10.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: Detected new or changed devices on smithi136 2024-02-10T06:56:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: Adjusting osd_memory_target on smithi136 to 17088M 2024-02-10T06:56:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:56:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:56:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:56:10.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:10 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: Detected new or changed devices on smithi136 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: Adjusting osd_memory_target on smithi136 to 17088M 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:56:10.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:10 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:11 smithi189 ceph-mon[30417]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:11 smithi189 ceph-mon[30417]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:11 smithi076 ceph-mon[25700]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:11.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:11 smithi076 ceph-mon[25700]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:11.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:11 smithi136 ceph-mon[29743]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:11.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:11 smithi136 ceph-mon[29743]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: from='client.? 172.21.15.189:0/3840262480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]': finished 2024-02-10T06:56:13.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:56:13.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:13 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: from='client.? 172.21.15.189:0/3840262480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]': finished 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:56:13.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:13 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: from='client.? 172.21.15.189:0/3840262480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]: dispatch 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05318e28-2ce4-4271-9f2d-220769fcf95c"}]': finished 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:56:13.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:13 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:14.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:14 smithi076 ceph-mon[25700]: from='client.? 172.21.15.189:0/596769791' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:56:14.572 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:14 smithi189 ceph-mon[30417]: from='client.? 172.21.15.189:0/596769791' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:56:14.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:14 smithi136 ceph-mon[29743]: from='client.? 172.21.15.189:0/596769791' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:56:15.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:15 smithi189 ceph-mon[30417]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:15.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:15 smithi076 ceph-mon[25700]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:15.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:15 smithi136 ceph-mon[29743]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:16.620 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:16 smithi189 ceph-mon[30417]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:16 smithi076 ceph-mon[25700]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:16.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:16 smithi136 ceph-mon[29743]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:19.490 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:19 smithi189 ceph-mon[30417]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:19.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:19 smithi076 ceph-mon[25700]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:19.589 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:19 smithi136 ceph-mon[29743]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:20.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:56:20.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:20 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:20.514 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:56:20.514 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:20 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:20.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:56:20.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:20 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:21.378 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:21 smithi189 ceph-mon[30417]: Deploying daemon osd.2 on smithi189 2024-02-10T06:56:21.379 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:21 smithi189 ceph-mon[30417]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:21.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:21 smithi076 ceph-mon[25700]: Deploying daemon osd.2 on smithi189 2024-02-10T06:56:21.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:21 smithi076 ceph-mon[25700]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:21.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:21 smithi136 ceph-mon[29743]: Deploying daemon osd.2 on smithi189 2024-02-10T06:56:21.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:21 smithi136 ceph-mon[29743]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:23.395 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:23 smithi189 ceph-mon[30417]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:23.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:23 smithi076 ceph-mon[25700]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:23.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:23 smithi136 ceph-mon[29743]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:25.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:25 smithi076 ceph-mon[25700]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:25.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:25 smithi136 ceph-mon[29743]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:25.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:25 smithi189 ceph-mon[30417]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:26.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:26 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:26 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:26 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:26.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:26 smithi136 ceph-mon[29743]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:26 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:26 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:26 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:26.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:26 smithi189 ceph-mon[30417]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:26 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:26 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:26 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:26.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:26 smithi076 ceph-mon[25700]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:29.175 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:29 smithi189 ceph-mon[30417]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.487 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.488 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:29 smithi076 ceph-mon[25700]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:29 smithi136 ceph-mon[29743]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:56:29.771 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.2.service 2024-02-10T06:56:29.773 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-10T06:56:29.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd stat -f json 2024-02-10T06:56:30.016 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:30.383 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-mon[30417]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:30.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:30 smithi076 ceph-mon[25700]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:30.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:30 smithi136 ceph-mon[29743]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-10T06:56:30.921 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2[35654]: 2024-02-10T06:56:30.668+0000 7f94727cc700 -1 osd.2 0 waiting for initial osdmap 2024-02-10T06:56:30.921 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:30 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2[35654]: 2024-02-10T06:56:30.673+0000 7f946a5b7700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:56:31.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:31.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-10T06:56:31.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:56:31.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:31 smithi189 ceph-mon[30417]: from='osd.2 ' entity='osd.2' 2024-02-10T06:56:31.431 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:31.431 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-10T06:56:31.431 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:56:31.431 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.431 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.432 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:31 smithi076 ceph-mon[25700]: from='osd.2 ' entity='osd.2' 2024-02-10T06:56:31.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:31.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-10T06:56:31.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:56:31.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:31.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:31 smithi136 ceph-mon[29743]: from='osd.2 ' entity='osd.2' 2024-02-10T06:56:31.868 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:56:32.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: purged_snaps scrub starts 2024-02-10T06:56:32.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: purged_snaps scrub ok 2024-02-10T06:56:32.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979] boot 2024-02-10T06:56:32.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:56:32.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:32 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3842711350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:56:32.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: purged_snaps scrub starts 2024-02-10T06:56:32.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: purged_snaps scrub ok 2024-02-10T06:56:32.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979] boot 2024-02-10T06:56:32.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:56:32.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:32 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3842711350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:56:32.485 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1707548191,"num_in_osds":3,"osd_in_since":1707548172,"num_remapped_pgs":0} 2024-02-10T06:56:32.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:56:32.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: purged_snaps scrub starts 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: purged_snaps scrub ok 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: osd.2 [v2:172.21.15.189:6800/153429979,v1:172.21.15.189:6801/153429979] boot 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:56:32.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:32 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3842711350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:56:32.719 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:33.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:33.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:33 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:33.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:33.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:33.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:33 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:33.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:33 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:34 smithi076 ceph-mon[25700]: Detected new or changed devices on smithi189 2024-02-10T06:56:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:34 smithi076 ceph-mon[25700]: Adjusting osd_memory_target on smithi189 to 17088M 2024-02-10T06:56:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:34.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:34 smithi076 ceph-mon[25700]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:56:34.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:34 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:56:34.606 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":20,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","created":"2024-02-10T06:53:10.466765+0000","modified":"2024-02-10T06:56:34.191982+0000","last_up_change":"2024-02-10T06:56:31.670856+0000","last_in_change":"2024-02-10T06:56:12.829402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:56:32.433763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39fc88c9-e027-42a1-8518-66357f266a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6803","nonce":1563965727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6805","nonce":1563965727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6809","nonce":1563965727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6807","nonce":1563965727}]},"public_addr":"172.21.15.76:6803/1563965727","cluster_addr":"172.21.15.76:6805/1563965727","heartbeat_back_addr":"172.21.15.76:6809/1563965727","heartbeat_front_addr":"172.21.15.76:6807/1563965727","state":["exists","up"]},{"osd":1,"uuid":"c2e47e83-c869-4489-90cb-91199a0901bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6801","nonce":3046476643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6803","nonce":3046476643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6807","nonce":3046476643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6805","nonce":3046476643}]},"public_addr":"172.21.15.136:6801/3046476643","cluster_addr":"172.21.15.136:6803/3046476643","heartbeat_back_addr":"172.21.15.136:6807/3046476643","heartbeat_front_addr":"172.21.15.136:6805/3046476643","state":["exists","up"]},{"osd":2,"uuid":"05318e28-2ce4-4271-9f2d-220769fcf95c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6801","nonce":153429979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6803","nonce":153429979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6807","nonce":153429979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6805","nonce":153429979}]},"public_addr":"172.21.15.189:6801/153429979","cluster_addr":"172.21.15.189:6803/153429979","heartbeat_back_addr":"172.21.15.189:6807/153429979","heartbeat_front_addr":"172.21.15.189:6805/153429979","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:55:41.088617+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:05.474952+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:29.455743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/3670878352":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6801/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/3083380644":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6800/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/675727288":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6800/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4008054001":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4168712414":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6801/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/3575996986":"2024-02-11T06:54:04.101784+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:56:34.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:34 smithi136 ceph-mon[29743]: Detected new or changed devices on smithi189 2024-02-10T06:56:34.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:34 smithi136 ceph-mon[29743]: Adjusting osd_memory_target on smithi189 to 17088M 2024-02-10T06:56:34.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:34 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:34.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:34 smithi136 ceph-mon[29743]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:56:34.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:34 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:34.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:34 smithi189 ceph-mon[30417]: Detected new or changed devices on smithi189 2024-02-10T06:56:34.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:34 smithi189 ceph-mon[30417]: Adjusting osd_memory_target on smithi189 to 17088M 2024-02-10T06:56:34.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:34 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:34.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:34 smithi189 ceph-mon[30417]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:56:34.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:34 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:34.796 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50458]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T06:56:34.796 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:34.796 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50458]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:34.796 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50458]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:35.155 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50473]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T06:56:35.155 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:35.156 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50473]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:35.156 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:56:34 smithi076 sudo[50473]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:35.182 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-10T06:56:32.433763+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-10T06:56:35.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-10T06:56:35.317 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39948]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T06:56:35.317 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39948]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:35.317 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39948]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:35.317 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39948]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:35.317 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:35.318 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:35.318 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:35.318 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:56:35.318 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/578028114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:56:35.318 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:35 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:35.419 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/578028114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:56:35.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:35 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/578028114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:56:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:35 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:35.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39951]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T06:56:35.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39951]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:35.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39951]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:35.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:56:35 smithi136 sudo[39951]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:35.840 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41228]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T06:56:35.840 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41228]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:35.840 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41228]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:35.840 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41228]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:36.171 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41231]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T06:56:36.171 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:36.171 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41231]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:36.171 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:56:35 smithi189 sudo[41231]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:36.364 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50598]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:56:36.364 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:36.364 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50598]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:36.364 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50598]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:36.629 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 ceph-mon[30417]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:56:36.629 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.629 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 ceph-mon[30417]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T06:56:36.629 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.630 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:36.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:36 smithi136 ceph-mon[29743]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:56:36.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:36 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:36 smithi136 ceph-mon[29743]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T06:56:36.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:36 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:36 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:36.702 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50609]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50609]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 sudo[50609]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 ceph-mon[25700]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 ceph-mon[25700]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:36.703 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:36 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:36.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41234]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:56:36.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:36.882 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41234]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:36.882 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41234]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:37.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41237]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:56:37.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:37.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41237]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:37.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:36 smithi189 sudo[41237]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:37.272 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-02-10T06:56:37.398 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39954]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:56:37.398 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:37.399 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39954]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:37.399 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39954]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: Cluster is now healthy 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.536 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:37 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/758910021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: Cluster is now healthy 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:37 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/758910021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39957]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39957]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 sudo[39957]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: Cluster is now healthy 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:56:37.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:37 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/758910021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:56:37.934 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi076 2024-02-10T06:56:37.935 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi076=foo.a' 2024-02-10T06:56:38.172 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:38.627 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:38 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:38.627 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:38 smithi189 ceph-mon[30417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:38.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:38 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:38.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:38 smithi189 ceph-mon[30417]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:38.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:38 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:38.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:38 smithi076 ceph-mon[25700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:38.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:38 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:38.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:38 smithi076 ceph-mon[25700]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:38.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:38 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:56:38.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:38 smithi136 ceph-mon[29743]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:56:38.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:38 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:38.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:38 smithi136 ceph-mon[29743]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:39.998 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled rgw.foo.a update... 2024-02-10T06:56:40.547 DEBUG:teuthology.orchestra.run.smithi076:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@rgw.foo.a.service 2024-02-10T06:56:40.549 INFO:tasks.cephadm:Adding prometheus.a on smithi136 2024-02-10T06:56:40.550 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply prometheus '1;smithi136=a' 2024-02-10T06:56:40.783 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:41.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi076=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:41.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:41.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:41 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:41.259 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi076=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.260 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:41 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi076=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:41.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:41 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:42.064 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:42 smithi189 ceph-mon[30417]: Deploying daemon rgw.foo.a on smithi076 2024-02-10T06:56:42.231 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:42 smithi076 ceph-mon[25700]: Deploying daemon rgw.foo.a on smithi076 2024-02-10T06:56:42.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:42 smithi136 ceph-mon[29743]: Deploying daemon rgw.foo.a on smithi076 2024-02-10T06:56:42.534 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled prometheus update... 2024-02-10T06:56:42.784 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:42 smithi076 systemd[1]: Starting Ceph rgw.foo.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:56:43.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:43 smithi189 ceph-mon[30417]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:43.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:43 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:43.209 DEBUG:teuthology.orchestra.run.smithi136:prometheus.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@prometheus.a.service 2024-02-10T06:56:43.212 INFO:tasks.cephadm:Adding node-exporter.a on smithi076 2024-02-10T06:56:43.212 INFO:tasks.cephadm:Adding node-exporter.b on smithi136 2024-02-10T06:56:43.212 INFO:tasks.cephadm:Adding node-exporter.c on smithi189 2024-02-10T06:56:43.213 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply node-exporter '3;smithi076=a;smithi136=b;smithi189=c' 2024-02-10T06:56:43.218 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:43 smithi076 ceph-mon[25700]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:43.218 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:43 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:43.218 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:42 smithi076 podman[51003]: 2024-02-10 06:56:42.802461291 +0000 UTC m=+0.018451842 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:56:43.218 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:42 smithi076 podman[51003]: 2024-02-10 06:56:42.908031119 +0000 UTC m=+0.124021711 container create 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-10T06:56:43.218 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:43 smithi076 podman[51003]: 2024-02-10 06:56:43.135084477 +0000 UTC m=+0.351075024 container init 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.vendor=CentOS) 2024-02-10T06:56:43.219 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:43 smithi076 podman[51003]: 2024-02-10 06:56:43.141194305 +0000 UTC m=+0.357184850 container start 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-10T06:56:43.219 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:43 smithi076 bash[51003]: 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 2024-02-10T06:56:43.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:43 smithi136 ceph-mon[29743]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:43.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:43 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:43.451 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:43.534 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:56:43 smithi076 systemd[1]: Started Ceph rgw.foo.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: Saving service prometheus spec with placement smithi136=a;count:1 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:44 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: Saving service prometheus spec with placement smithi136=a;count:1 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:44 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: Saving service prometheus spec with placement smithi136=a;count:1 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:44.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:44 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:45.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:45.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T06:56:45.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:56:45.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: pgmap v92: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:45.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:45.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:45 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T06:56:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:56:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: pgmap v92: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:45.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:45.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:45 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:45.352 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled node-exporter update... 2024-02-10T06:56:45.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: Saving service rgw.foo.a spec with placement smithi076=foo.a;count:1 2024-02-10T06:56:45.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: pgmap v92: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:45.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:45 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:46.026 DEBUG:teuthology.orchestra.run.smithi076:node-exporter.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.a.service 2024-02-10T06:56:46.028 DEBUG:teuthology.orchestra.run.smithi136:node-exporter.b> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.b.service 2024-02-10T06:56:46.031 DEBUG:teuthology.orchestra.run.smithi189:node-exporter.c> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.c.service 2024-02-10T06:56:46.033 INFO:tasks.cephadm:Adding alertmanager.a on smithi076 2024-02-10T06:56:46.033 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply alertmanager '1;smithi076=a' 2024-02-10T06:56:46.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:46 smithi076 ceph-mon[25700]: Checking dashboard <-> RGW credentials 2024-02-10T06:56:46.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:46 smithi076 ceph-mon[25700]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:46.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:46 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:56:46.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:46 smithi076 ceph-mon[25700]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T06:56:46.285 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:46 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:46.312 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:46.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:46 smithi136 ceph-mon[29743]: Checking dashboard <-> RGW credentials 2024-02-10T06:56:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:46 smithi136 ceph-mon[29743]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:46 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:56:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:46 smithi136 ceph-mon[29743]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T06:56:46.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:46 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:46 smithi189 ceph-mon[30417]: Checking dashboard <-> RGW credentials 2024-02-10T06:56:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:46 smithi189 ceph-mon[30417]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:46 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/15452119' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:56:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:46 smithi189 ceph-mon[30417]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T06:56:46.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:46 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi076=a;smithi136=b;smithi189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: Saving service node-exporter spec with placement smithi076=a;smithi136=b;smithi189=c;count:3 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:47 smithi136 ceph-mon[29743]: pgmap v95: 65 pgs: 8 creating+peering, 1 active+clean, 56 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi076=a;smithi136=b;smithi189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: Saving service node-exporter spec with placement smithi076=a;smithi136=b;smithi189=c;count:3 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.414 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:47 smithi189 ceph-mon[30417]: pgmap v95: 65 pgs: 8 creating+peering, 1 active+clean, 56 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi076=a;smithi136=b;smithi189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: Saving service node-exporter spec with placement smithi076=a;smithi136=b;smithi189=c;count:3 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:56:47.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:47 smithi076 ceph-mon[25700]: pgmap v95: 65 pgs: 8 creating+peering, 1 active+clean, 56 unknown; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:56:48.210 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled alertmanager update... 2024-02-10T06:56:48.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:48 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:48 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:48 smithi136 ceph-mon[29743]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T06:56:48.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:48 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:48 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:48 smithi189 ceph-mon[30417]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T06:56:48.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:48 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:48 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:56:48.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:48 smithi076 ceph-mon[25700]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T06:56:48.830 DEBUG:teuthology.orchestra.run.smithi076:alertmanager.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@alertmanager.a.service 2024-02-10T06:56:48.834 INFO:tasks.cephadm:Adding grafana.a on smithi136 2024-02-10T06:56:48.834 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph orch apply grafana '1;smithi136=a' 2024-02-10T06:56:49.075 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: pgmap v98: 97 pgs: 8 creating+peering, 24 active+clean, 65 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi076=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: Saving service alertmanager spec with placement smithi076=a;count:1 2024-02-10T06:56:49.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:49 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:49.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: pgmap v98: 97 pgs: 8 creating+peering, 24 active+clean, 65 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi076=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: Saving service alertmanager spec with placement smithi076=a;count:1 2024-02-10T06:56:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:49 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:49.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T06:56:49.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:56:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: pgmap v98: 97 pgs: 8 creating+peering, 24 active+clean, 65 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-02-10T06:56:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi076=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: Saving service alertmanager spec with placement smithi076=a;count:1 2024-02-10T06:56:49.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:49 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:50.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:50 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.167 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:50 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.167 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:50 smithi189 ceph-mon[30417]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T06:56:50.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:50 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:50 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:50 smithi136 ceph-mon[29743]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T06:56:50.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:50 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:50 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:56:50.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:50 smithi076 ceph-mon[25700]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T06:56:50.878 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled grafana update... 2024-02-10T06:56:51.155 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T06:56:51.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: pgmap v101: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T06:56:51.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:51.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:51 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: pgmap v101: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:51.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:51 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:51.465 DEBUG:teuthology.orchestra.run.smithi136:grafana.a> sudo journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@grafana.a.service 2024-02-10T06:56:51.468 INFO:tasks.cephadm:Setting up client nodes... 2024-02-10T06:56:51.469 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: pgmap v101: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:51.497 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:51 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:51.718 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:56:52.124 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:52.124 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: Saving service grafana spec with placement smithi136=a;count:1 2024-02-10T06:56:52.124 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:52.124 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T06:56:52.125 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:52.125 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:52 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:52.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:52.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: Saving service grafana spec with placement smithi136=a;count:1 2024-02-10T06:56:52.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:52.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T06:56:52.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:52.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:52 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi136=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: Saving service grafana spec with placement smithi136=a;count:1 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:52.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:52 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:56:53.138 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.138 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.139 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T06:56:53.139 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: pgmap v104: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:56:53.139 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.139 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.139 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:53 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: pgmap v104: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:53 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/835607253' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3025848905' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: pgmap v104: 129 pgs: 20 creating+peering, 65 active+clean, 44 unknown; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:53 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:56:53.669 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-02-10T06:56:53.669 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQA1HsdlGvLHJxAAL8SO+e9ipn3ZifClcwHnfw== 2024-02-10T06:56:54.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:54 smithi076 ceph-mon[25700]: Deploying daemon prometheus.a on smithi136 2024-02-10T06:56:54.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:54 smithi076 ceph-mon[25700]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:54.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:54 smithi076 ceph-mon[25700]: Cluster is now healthy 2024-02-10T06:56:54.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:54 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:56:54.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:54 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:56:54.384 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:56:54.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-10T06:56:54.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-10T06:56:54.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:54 smithi136 ceph-mon[29743]: Deploying daemon prometheus.a on smithi136 2024-02-10T06:56:54.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:54 smithi136 ceph-mon[29743]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:54.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:54 smithi136 ceph-mon[29743]: Cluster is now healthy 2024-02-10T06:56:54.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:54 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:56:54.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:54 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:56:54.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:54 smithi189 ceph-mon[30417]: Deploying daemon prometheus.a on smithi136 2024-02-10T06:56:54.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:54 smithi189 ceph-mon[30417]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:56:54.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:54 smithi189 ceph-mon[30417]: Cluster is now healthy 2024-02-10T06:56:54.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:54 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:56:54.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:54 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3932870002' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:56:54.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T06:56:54.724 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:56:55.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:55 smithi136 ceph-mon[29743]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 7.1 KiB/s wr, 187 op/s 2024-02-10T06:56:55.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:55 smithi189 ceph-mon[30417]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 7.1 KiB/s wr, 187 op/s 2024-02-10T06:56:55.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:55 smithi076 ceph-mon[25700]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 75 KiB/s rd, 7.1 KiB/s wr, 187 op/s 2024-02-10T06:56:56.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:56 smithi136 ceph-mon[29743]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 5.9 KiB/s wr, 156 op/s 2024-02-10T06:56:56.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:56 smithi189 ceph-mon[30417]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 5.9 KiB/s wr, 156 op/s 2024-02-10T06:56:57.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:56 smithi076 ceph-mon[25700]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 63 KiB/s rd, 5.9 KiB/s wr, 156 op/s 2024-02-10T06:56:59.322 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:56:59 smithi136 ceph-mon[29743]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T06:56:59.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:56:59 smithi076 ceph-mon[25700]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T06:56:59.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:56:59 smithi189 ceph-mon[30417]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T06:56:59.912 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2024-02-10T06:56:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA7Hsdl7rAVNhAAObC8Ncdc51YGdwd1oKDOIw== 2024-02-10T06:57:00.480 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:00 smithi136 ceph-mon[29743]: from='client.? 172.21.15.136:0/4090796775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.480 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:00 smithi136 ceph-mon[29743]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.480 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:00 smithi136 ceph-mon[29743]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:00.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:00 smithi076 ceph-mon[25700]: from='client.? 172.21.15.136:0/4090796775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:00 smithi076 ceph-mon[25700]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:00 smithi076 ceph-mon[25700]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:00.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:00 smithi189 ceph-mon[30417]: from='client.? 172.21.15.136:0/4090796775' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:00 smithi189 ceph-mon[30417]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:00.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:00 smithi189 ceph-mon[30417]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:00.781 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:57:00.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-10T06:57:00.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-10T06:57:00.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T06:57:01.070 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T06:57:01.368 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:01 smithi136 ceph-mon[29743]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 4.0 KiB/s wr, 124 op/s 2024-02-10T06:57:01.449 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:01 smithi189 ceph-mon[30417]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 4.0 KiB/s wr, 124 op/s 2024-02-10T06:57:01.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:01 smithi076 ceph-mon[25700]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 51 KiB/s rd, 4.0 KiB/s wr, 124 op/s 2024-02-10T06:57:02.986 INFO:teuthology.orchestra.run.smithi189.stdout:[client.2] 2024-02-10T06:57:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQA+Hsdl94CIOhAA/nPe64WZ9LEQL9yM9FMiew== 2024-02-10T06:57:03.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:03 smithi189 ceph-mon[30417]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T06:57:03.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:03 smithi189 ceph-mon[30417]: from='client.? 172.21.15.189:0/4235524552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:03 smithi189 ceph-mon[30417]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:03 smithi189 ceph-mon[30417]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:03.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:03 smithi076 ceph-mon[25700]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T06:57:03.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:03 smithi076 ceph-mon[25700]: from='client.? 172.21.15.189:0/4235524552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:03 smithi076 ceph-mon[25700]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:03 smithi076 ceph-mon[25700]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-mon[29743]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T06:57:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-mon[29743]: from='client.? 172.21.15.189:0/4235524552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-mon[29743]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:57:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-mon[29743]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:57:03.554 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 systemd[1]: Starting Ceph prometheus.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:57:03.592 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:57:03.592 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-10T06:57:03.592 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-10T06:57:03.634 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-10T06:57:03.635 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-10T06:57:03.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph mgr dump --format=json 2024-02-10T06:57:03.809 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 podman[40875]: 2024-02-10 06:57:03.553196272 +0000 UTC m=+0.103582535 container create e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:03.809 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 podman[40875]: 2024-02-10 06:57:03.469231516 +0000 UTC m=+0.019617717 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-10T06:57:03.809 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 podman[40875]: 2024-02-10 06:57:03.780359568 +0000 UTC m=+0.330745770 container init e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:03.810 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 podman[40875]: 2024-02-10 06:57:03.785368665 +0000 UTC m=+0.335754867 container start e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:03.881 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:04.136 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.809Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-10T06:57:04.136 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.809Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-10T06:57:04.136 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.809Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi136 (none))" 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.809Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.809Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.810Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.811Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.811Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.811Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.813Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.813Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.933µs 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.813Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.814Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.814Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=29.436µs wal_replay_duration=507.314µs wbl_replay_duration=138ns total_replay_duration=612.167µs 2024-02-10T06:57:04.137 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.815Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.815Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.815Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.831Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.639846ms db_storage=659ns remote_storage=1.058µs web_handler=460ns query_engine=493ns scrape=391.49µs scrape_sd=67.095µs notify=707ns notify_sd=1.228µs rules=16.001667ms tracing=9.499µs 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.832Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:57:03.832Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 bash[40875]: e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d 2024-02-10T06:57:04.138 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:57:03 smithi136 systemd[1]: Started Ceph prometheus.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:57:05.005 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.005 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.006 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.006 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:57:05.006 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-10T06:57:05.006 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:04 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.262 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setuser ceph since I am not root 2024-02-10T06:57:05.262 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ignoring --setgroup ceph since I am not root 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-10T06:57:05.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:04 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.387 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: ignoring --setuser ceph since I am not root 2024-02-10T06:57:05.387 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: ignoring --setgroup ceph since I am not root 2024-02-10T06:57:05.387 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-mgr[31214]: -- 172.21.15.136:0/215977617 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dd000ff080 con 0x55dd000c8800 2024-02-10T06:57:05.387 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-mgr[31214]: -- 172.21.15.136:0/215977617 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dd000f21a0 con 0x55dd000c8800 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 93 op/s 2024-02-10T06:57:05.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:04 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' 2024-02-10T06:57:05.422 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: ignoring --setuser ceph since I am not root 2024-02-10T06:57:05.422 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: ignoring --setgroup ceph since I am not root 2024-02-10T06:57:05.422 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-mgr[32508]: -- 172.21.15.189:0/1684466059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622a20b41a0 con 0x5622a208ac00 2024-02-10T06:57:05.885 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:05.518+0000 7faa12daf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:57:05.885 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:05.610+0000 7faa12daf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:57:05.920 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:05.519+0000 7feab9029200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:57:05.921 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:05.613+0000 7feab9029200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:57:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:05.938 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:05.552+0000 7f166d424200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:57:05.938 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:05.649+0000 7f166d424200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:57:06.240 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-mon[29743]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:57:06.240 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-mon[29743]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T06:57:06.240 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:05 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/2497604788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:57:06.244 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-mon[25700]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:57:06.244 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-mon[25700]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T06:57:06.245 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:05 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2497604788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:57:06.248 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-mon[30417]: from='mgr.14152 172.21.15.76:0/2485725438' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:57:06.248 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-mon[30417]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T06:57:06.248 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:05 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/2497604788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:57:06.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:06.306+0000 7f166d424200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:57:06.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:06.400+0000 7f166d424200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:57:06.568 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.239+0000 7faa12daf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:57:06.568 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.332+0000 7faa12daf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:57:06.568 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.471+0000 7faa12daf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:57:06.568 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.567+0000 7faa12daf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:57:06.578 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.248+0000 7feab9029200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:57:06.579 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.341+0000 7feab9029200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:57:06.579 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.482+0000 7feab9029200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:57:06.579 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.578+0000 7feab9029200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:57:06.597 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":3207057037},{"type":"v1","addr":"172.21.15.76:6801","nonce":3207057037}]},"active_addr":"172.21.15.76:6801/3207057037","active_change":"2024-02-10T06:54:04.101900+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.76:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1658822003}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":3310817264}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1713441226}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":14908455}]}]} 2024-02-10T06:57:06.603 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-10T06:57:06.603 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-10T06:57:06.603 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:57:06.848 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:06.885 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.650+0000 7faa12daf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:57:06.921 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.661+0000 7feab9029200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:57:06.969 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:06.545+0000 7f166d424200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:57:06.969 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:06.645+0000 7f166d424200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:57:06.970 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:06.731+0000 7f166d424200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:57:07.202 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:06 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:06.912+0000 7faa12daf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:57:07.202 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:07 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:07.019+0000 7faa12daf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:07.202 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:07 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:07.105+0000 7faa12daf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:57:07.202 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:07 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:07.200+0000 7faa12daf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:57:07.210 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:06 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:06.922+0000 7feab9029200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:57:07.210 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:07 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:07.030+0000 7feab9029200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:07.210 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:07 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:07.116+0000 7feab9029200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:57:07.210 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:07 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:07.210+0000 7feab9029200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:57:07.255 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:07 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:07.011+0000 7f166d424200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:57:07.256 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:07 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:07.122+0000 7f166d424200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:07.256 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:07 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:07.213+0000 7f166d424200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:57:07.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:07 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:07.311+0000 7f166d424200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:57:07.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:07 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:07.487+0000 7f166d424200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:57:07.635 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:07 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:07.376+0000 7faa12daf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:57:07.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:07 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:07.387+0000 7feab9029200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:57:08.635 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:08 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:08.329+0000 7faa12daf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:57:08.636 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:08 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:08.459+0000 7faa12daf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:57:08.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:08 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:08.347+0000 7feab9029200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:57:08.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:08 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:08.476+0000 7feab9029200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:57:08.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:08 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:08.466+0000 7f166d424200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:57:08.784 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:08 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:08.603+0000 7f166d424200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:57:08.859 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:08.859 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":30,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","created":"2024-02-10T06:53:10.466765+0000","modified":"2024-02-10T06:56:52.066508+0000","last_up_change":"2024-02-10T06:56:31.670856+0000","last_in_change":"2024-02-10T06:56:12.829402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:56:32.433763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T06:56:43.413970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-10T06:56:45.330634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-10T06:56:47.290544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-10T06:56:49.359805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39fc88c9-e027-42a1-8518-66357f266a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6803","nonce":1563965727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6805","nonce":1563965727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6809","nonce":1563965727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6807","nonce":1563965727}]},"public_addr":"172.21.15.76:6803/1563965727","cluster_addr":"172.21.15.76:6805/1563965727","heartbeat_back_addr":"172.21.15.76:6809/1563965727","heartbeat_front_addr":"172.21.15.76:6807/1563965727","state":["exists","up"]},{"osd":1,"uuid":"c2e47e83-c869-4489-90cb-91199a0901bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6801","nonce":3046476643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6803","nonce":3046476643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6807","nonce":3046476643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6805","nonce":3046476643}]},"public_addr":"172.21.15.136:6801/3046476643","cluster_addr":"172.21.15.136:6803/3046476643","heartbeat_back_addr":"172.21.15.136:6807/3046476643","heartbeat_front_addr":"172.21.15.136:6805/3046476643","state":["exists","up"]},{"osd":2,"uuid":"05318e28-2ce4-4271-9f2d-220769fcf95c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6801","nonce":153429979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6803","nonce":153429979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6807","nonce":153429979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6805","nonce":153429979}]},"public_addr":"172.21.15.189:6801/153429979","cluster_addr":"172.21.15.189:6803/153429979","heartbeat_back_addr":"172.21.15.189:6807/153429979","heartbeat_front_addr":"172.21.15.189:6805/153429979","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:55:41.088617+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:05.474952+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:29.455743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/3670878352":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6801/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/3083380644":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6800/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/675727288":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6800/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4008054001":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4168712414":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6801/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/3575996986":"2024-02-11T06:54:04.101784+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:57:09.088 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:08 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/1047637958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:09.088 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:08 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:08.800+0000 7faa12daf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:57:09.088 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:08 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:08.891+0000 7faa12daf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:57:09.088 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:09.087+0000 7faa12daf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:09.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:08 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/1047637958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:09.096 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:08 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:08.811+0000 7feab9029200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:57:09.097 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:08 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:08.900+0000 7feab9029200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:57:09.097 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:09 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:09.096+0000 7feab9029200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:09.231 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:08 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/1047637958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:09.243 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:08 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:08.970+0000 7f166d424200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:57:09.243 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:09 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:09.061+0000 7f166d424200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:57:09.385 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:09.271+0000 7faa12daf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:57:09.421 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:09 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:09.280+0000 7feab9029200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:57:09.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:09 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:09.264+0000 7f166d424200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:09.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:09 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:09.450+0000 7f166d424200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:57:09.568 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-10T06:57:09.568 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:57:09.815 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:09.555+0000 7faa12daf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:09.815 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:09.814+0000 7faa12daf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:09.818 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:09.821 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:09 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:09.561+0000 7feab9029200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:09.821 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:09 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:09.821+0000 7feab9029200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:09.940 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:09 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:09.747+0000 7f166d424200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:57:10.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:10 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:10.048+0000 7faa12daf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:57:10.171 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:10 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:10.055+0000 7feab9029200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:57:10.209 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:10 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:10.020+0000 7f166d424200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:57:10.534 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:10 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:10.269+0000 7f166d424200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:57:11.135 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:10 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:10.843+0000 7faa12daf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:57:11.171 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:10 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:10.846+0000 7feab9029200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:57:11.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:11.106+0000 7f166d424200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:57:11.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:11.419+0000 7feab9029200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:57:11.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:11.518+0000 7feab9029200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:57:11.727 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:11.413+0000 7faa12daf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:57:11.727 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:11.512+0000 7faa12daf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:57:11.727 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:11.726+0000 7faa12daf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:57:11.826 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:11.826 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":30,"fsid":"e8085918-c7e0-11ee-95b8-87774f69a715","created":"2024-02-10T06:53:10.466765+0000","modified":"2024-02-10T06:56:52.066508+0000","last_up_change":"2024-02-10T06:56:31.670856+0000","last_in_change":"2024-02-10T06:56:12.829402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:56:32.433763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T06:56:43.413970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-10T06:56:45.330634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-10T06:56:47.290544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-10T06:56:49.359805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39fc88c9-e027-42a1-8518-66357f266a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6803","nonce":1563965727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6805","nonce":1563965727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6809","nonce":1563965727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1563965727},{"type":"v1","addr":"172.21.15.76:6807","nonce":1563965727}]},"public_addr":"172.21.15.76:6803/1563965727","cluster_addr":"172.21.15.76:6805/1563965727","heartbeat_back_addr":"172.21.15.76:6809/1563965727","heartbeat_front_addr":"172.21.15.76:6807/1563965727","state":["exists","up"]},{"osd":1,"uuid":"c2e47e83-c869-4489-90cb-91199a0901bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6801","nonce":3046476643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6803","nonce":3046476643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6807","nonce":3046476643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3046476643},{"type":"v1","addr":"172.21.15.136:6805","nonce":3046476643}]},"public_addr":"172.21.15.136:6801/3046476643","cluster_addr":"172.21.15.136:6803/3046476643","heartbeat_back_addr":"172.21.15.136:6807/3046476643","heartbeat_front_addr":"172.21.15.136:6805/3046476643","state":["exists","up"]},{"osd":2,"uuid":"05318e28-2ce4-4271-9f2d-220769fcf95c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6801","nonce":153429979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6803","nonce":153429979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6807","nonce":153429979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":153429979},{"type":"v1","addr":"172.21.15.189:6805","nonce":153429979}]},"public_addr":"172.21.15.189:6801/153429979","cluster_addr":"172.21.15.189:6803/153429979","heartbeat_back_addr":"172.21.15.189:6807/153429979","heartbeat_front_addr":"172.21.15.189:6805/153429979","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:55:41.088617+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:05.474952+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:56:29.455743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/3670878352":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6801/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/3083380644":"2024-02-11T06:54:04.101784+0000","172.21.15.76:6800/1701799245":"2024-02-11T06:54:04.101784+0000","172.21.15.76:0/675727288":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6800/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4008054001":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/4168712414":"2024-02-11T06:53:37.253961+0000","172.21.15.76:6801/2849555438":"2024-02-11T06:53:37.253961+0000","172.21.15.76:0/3575996986":"2024-02-11T06:54:04.101784+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:57:12.022 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:11.694+0000 7f166d424200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:57:12.022 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:11.799+0000 7f166d424200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: Standby manager daemon x restarted 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: Standby manager daemon x started 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: Standby manager daemon z restarted 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: Standby manager daemon z started 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:57:12.023 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:11 smithi076 ceph-mon[25700]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: [10/Feb/2024:06:57:11] ENGINE Bus STARTING 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: CherryPy Checker: 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: The Application mounted at '' has an empty config. 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: [10/Feb/2024:06:57:11] ENGINE Serving on http://:::9283 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x[31191]: [10/Feb/2024:06:57:11] ENGINE Bus STARTED 2024-02-10T06:57:12.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: Standby manager daemon x restarted 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: Standby manager daemon x started 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: Standby manager daemon z restarted 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: Standby manager daemon z started 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:57:12.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:11 smithi136 ceph-mon[29743]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: Standby manager daemon x restarted 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: Standby manager daemon x started 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:57:12.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.136:0/792399664' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: Standby manager daemon z restarted 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: Standby manager daemon z started 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-mon[30417]: from='mgr.? 172.21.15.189:0/1418204512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:11.733+0000 7feab9029200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: [10/Feb/2024:06:57:11] ENGINE Bus STARTING 2024-02-10T06:57:12.172 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: CherryPy Checker: 2024-02-10T06:57:12.173 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: The Application mounted at '' has an empty config. 2024-02-10T06:57:12.173 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: 2024-02-10T06:57:12.173 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: [10/Feb/2024:06:57:11] ENGINE Serving on http://:::9283 2024-02-10T06:57:12.173 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:57:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z[32484]: [10/Feb/2024:06:57:11] ENGINE Bus STARTED 2024-02-10T06:57:12.274 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:12.021+0000 7f166d424200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:57:12.274 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:12] ENGINE Bus STARTING 2024-02-10T06:57:12.274 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: CherryPy Checker: 2024-02-10T06:57:12.274 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: The Application mounted at '' has an empty config. 2024-02-10T06:57:12.274 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:57:12.275 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:12] ENGINE Serving on http://:::9283 2024-02-10T06:57:12.275 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:12] ENGINE Bus STARTED 2024-02-10T06:57:12.516 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-10T06:57:12.517 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-10T06:57:12.517 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-10T06:57:12.794 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/820676370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: Active manager daemon y restarted 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: Activating manager daemon y 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T06:57:12.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: mgrmap e19: y(active, starting, since 0.0111251s), standbys: x, z 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:57:12.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: Manager daemon y is now available 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:57:12.807 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:12 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/820676370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: Active manager daemon y restarted 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: Activating manager daemon y 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: mgrmap e19: y(active, starting, since 0.0111251s), standbys: x, z 2024-02-10T06:57:13.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: Manager daemon y is now available 2024-02-10T06:57:13.137 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:13.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:13.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:57:13.138 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:12 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/820676370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: Active manager daemon y restarted 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: Activating manager daemon y 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: mgrmap e19: y(active, starting, since 0.0111251s), standbys: x, z 2024-02-10T06:57:13.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: Manager daemon y is now available 2024-02-10T06:57:13.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:13.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:13.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:13.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:57:13.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:12 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:57:13.455 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:13.455 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:14.173 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: mgrmap e20: y(active, since 1.01796s), standbys: x, z 2024-02-10T06:57:14.173 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: [10/Feb/2024:06:57:13] ENGINE Bus STARTING 2024-02-10T06:57:14.173 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: [10/Feb/2024:06:57:13] ENGINE Serving on https://172.21.15.76:7150 2024-02-10T06:57:14.174 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: [10/Feb/2024:06:57:13] ENGINE Serving on http://172.21.15.76:8765 2024-02-10T06:57:14.174 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: [10/Feb/2024:06:57:13] ENGINE Bus STARTED 2024-02-10T06:57:14.174 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:14.174 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:14 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:14.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: mgrmap e20: y(active, since 1.01796s), standbys: x, z 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: [10/Feb/2024:06:57:13] ENGINE Bus STARTING 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: [10/Feb/2024:06:57:13] ENGINE Serving on https://172.21.15.76:7150 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: [10/Feb/2024:06:57:13] ENGINE Serving on http://172.21.15.76:8765 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: [10/Feb/2024:06:57:13] ENGINE Bus STARTED 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:14.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:14 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: mgrmap e20: y(active, since 1.01796s), standbys: x, z 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: [10/Feb/2024:06:57:13] ENGINE Bus STARTING 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: [10/Feb/2024:06:57:13] ENGINE Serving on https://172.21.15.76:7150 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: [10/Feb/2024:06:57:13] ENGINE Serving on http://172.21.15.76:8765 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: [10/Feb/2024:06:57:13] ENGINE Bus STARTED 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:14.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:14 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:15.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:15 smithi189 ceph-mon[30417]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:15 smithi076 ceph-mon[25700]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-10T06:57:15.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:15.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:15.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:15 smithi136 ceph-mon[29743]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-10T06:57:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-mon[25700]: Adjusting osd_memory_target on smithi136 to 16064M 2024-02-10T06:57:16.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:16 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:16 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:16 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:16 smithi136 ceph-mon[29743]: Adjusting osd_memory_target on smithi136 to 16064M 2024-02-10T06:57:16.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:16 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:16 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:16 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:16.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:16 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:16.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:16 smithi189 ceph-mon[30417]: Adjusting osd_memory_target on smithi136 to 16064M 2024-02-10T06:57:16.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:16 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:17.268 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:16 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:57:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:77309411339 2024-02-10T06:57:17.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-10T06:57:17.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:17 smithi136 ceph-mon[29743]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:17.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:17 smithi136 ceph-mon[29743]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-10T06:57:17.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:17 smithi189 ceph-mon[30417]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:17.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:17 smithi189 ceph-mon[30417]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-10T06:57:18.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:17 smithi076 ceph-mon[25700]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:18.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:17 smithi076 ceph-mon[25700]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-10T06:57:19.417 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:19 smithi076 ceph-mon[25700]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:19.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:19 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.418 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:19 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:19 smithi136 ceph-mon[29743]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:19.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:19 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:19 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:19 smithi189 ceph-mon[30417]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:19.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:19 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:19 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:34359738390 2024-02-10T06:57:19.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-10T06:57:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:55834574865 2024-02-10T06:57:20.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-10T06:57:20.204 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:20.998 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:21.004 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:21.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T06:57:21.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Adjusting osd_memory_target on smithi076 to 16064M 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:21 smithi076 ceph-mon[25700]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Adjusting osd_memory_target on smithi076 to 16064M 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:21 smithi136 ceph-mon[29743]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Adjusting osd_memory_target on smithi076 to 16064M 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:21.924 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-10T06:57:21.925 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi136:/etc/ceph/ceph.conf 2024-02-10T06:57:21.925 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-10T06:57:21.925 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.925 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:21.925 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:21 smithi189 ceph-mon[30417]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:57:22.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.785 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.786 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.786 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:22 smithi076 ceph-mon[25700]: Deploying daemon alertmanager.a on smithi076 2024-02-10T06:57:22.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.887 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:22 smithi136 ceph-mon[29743]: Deploying daemon alertmanager.a on smithi076 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Updating smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:57:22.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:22.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:22 smithi189 ceph-mon[30417]: Deploying daemon alertmanager.a on smithi076 2024-02-10T06:57:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:34359738390 2024-02-10T06:57:23.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:23 smithi136 ceph-mon[29743]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T06:57:23.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:23 smithi189 ceph-mon[30417]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T06:57:24.010 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:23 smithi076 ceph-mon[25700]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T06:57:24.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738390 got 34359738390 for osd.0 2024-02-10T06:57:24.358 DEBUG:teuthology.parallel:result is None 2024-02-10T06:57:24.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:24 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/3297014684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:57:24.917 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:24 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/3297014684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:57:24.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:24 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/3297014684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:57:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:77309411341 2024-02-10T06:57:25.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:25 smithi136 ceph-mon[29743]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:57:25.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:25 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/2072607255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:57:25.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:25 smithi189 ceph-mon[30417]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:57:25.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:25 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/2072607255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:57:26.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:25 smithi076 ceph-mon[25700]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:57:26.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:25 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/2072607255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:57:26.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411341 for osd.2 2024-02-10T06:57:26.316 DEBUG:teuthology.parallel:result is None 2024-02-10T06:57:27.186 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:26 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:57:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:55834574866 2024-02-10T06:57:27.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:27 smithi136 ceph-mon[29743]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:27.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:27 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:27.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:27 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/271322078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:57:27.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:27 smithi189 ceph-mon[30417]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:27.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:27 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:27.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:27 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/271322078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:57:28.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:27 smithi076 ceph-mon[25700]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:28.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:27 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:28.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:27 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/271322078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:57:28.620 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574865 got 55834574866 for osd.1 2024-02-10T06:57:28.620 DEBUG:teuthology.parallel:result is None 2024-02-10T06:57:28.620 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-10T06:57:28.621 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T06:57:28.916 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:29.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:29 smithi076 ceph-mon[25700]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:29.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:29 smithi136 ceph-mon[29743]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:29.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:29 smithi189 ceph-mon[30417]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:31.858 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:31 smithi076 ceph-mon[25700]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:31.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:31 smithi136 ceph-mon[29743]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:31.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:31 smithi189 ceph-mon[30417]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T06:57:32.723 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:32.726 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-02-10T06:57:33.579 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-02-10T06:57:32.055881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96616,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126552,"statfs":{"total":287972524032,"available":287873589248,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043001+0000","last_change":"2024-02-10T06:56:51.075264+0000","last_active":"2024-02-10T06:57:12.043001+0000","last_peered":"2024-02-10T06:57:12.043001+0000","last_clean":"2024-02-10T06:57:12.043001+0000","last_became_active":"2024-02-10T06:56:51.074593+0000","last_became_peered":"2024-02-10T06:56:51.074593+0000","last_unstale":"2024-02-10T06:57:12.043001+0000","last_undegraded":"2024-02-10T06:57:12.043001+0000","last_fullsized":"2024-02-10T06:57:12.043001+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:04:11.432491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045667+0000","last_change":"2024-02-10T06:56:49.072725+0000","last_active":"2024-02-10T06:57:12.045667+0000","last_peered":"2024-02-10T06:57:12.045667+0000","last_clean":"2024-02-10T06:57:12.045667+0000","last_became_active":"2024-02-10T06:56:49.072454+0000","last_became_peered":"2024-02-10T06:56:49.072454+0000","last_unstale":"2024-02-10T06:57:12.045667+0000","last_undegraded":"2024-02-10T06:57:12.045667+0000","last_fullsized":"2024-02-10T06:57:12.045667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:19:09.392591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045389+0000","last_change":"2024-02-10T06:56:45.034172+0000","last_active":"2024-02-10T06:57:12.045389+0000","last_peered":"2024-02-10T06:57:12.045389+0000","last_clean":"2024-02-10T06:57:12.045389+0000","last_became_active":"2024-02-10T06:56:45.033396+0000","last_became_peered":"2024-02-10T06:56:45.033396+0000","last_unstale":"2024-02-10T06:57:12.045389+0000","last_undegraded":"2024-02-10T06:57:12.045389+0000","last_fullsized":"2024-02-10T06:57:12.045389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:52.432749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044316+0000","last_change":"2024-02-10T06:56:47.055864+0000","last_active":"2024-02-10T06:57:12.044316+0000","last_peered":"2024-02-10T06:57:12.044316+0000","last_clean":"2024-02-10T06:57:12.044316+0000","last_became_active":"2024-02-10T06:56:47.055532+0000","last_became_peered":"2024-02-10T06:56:47.055532+0000","last_unstale":"2024-02-10T06:57:12.044316+0000","last_undegraded":"2024-02-10T06:57:12.044316+0000","last_fullsized":"2024-02-10T06:57:12.044316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:06:28.347666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041863+0000","last_change":"2024-02-10T06:56:51.077080+0000","last_active":"2024-02-10T06:57:12.041863+0000","last_peered":"2024-02-10T06:57:12.041863+0000","last_clean":"2024-02-10T06:57:12.041863+0000","last_became_active":"2024-02-10T06:56:51.076740+0000","last_became_peered":"2024-02-10T06:56:51.076740+0000","last_unstale":"2024-02-10T06:57:12.041863+0000","last_undegraded":"2024-02-10T06:57:12.041863+0000","last_fullsized":"2024-02-10T06:57:12.041863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:01:13.462733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042561+0000","last_change":"2024-02-10T06:56:49.071291+0000","last_active":"2024-02-10T06:57:12.042561+0000","last_peered":"2024-02-10T06:57:12.042561+0000","last_clean":"2024-02-10T06:57:12.042561+0000","last_became_active":"2024-02-10T06:56:49.070999+0000","last_became_peered":"2024-02-10T06:56:49.070999+0000","last_unstale":"2024-02-10T06:57:12.042561+0000","last_undegraded":"2024-02-10T06:57:12.042561+0000","last_fullsized":"2024-02-10T06:57:12.042561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:55:50.222415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044493+0000","last_change":"2024-02-10T06:56:45.034169+0000","last_active":"2024-02-10T06:57:12.044493+0000","last_peered":"2024-02-10T06:57:12.044493+0000","last_clean":"2024-02-10T06:57:12.044493+0000","last_became_active":"2024-02-10T06:56:45.033138+0000","last_became_peered":"2024-02-10T06:56:45.033138+0000","last_unstale":"2024-02-10T06:57:12.044493+0000","last_undegraded":"2024-02-10T06:57:12.044493+0000","last_fullsized":"2024-02-10T06:57:12.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:47:59.506891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045136+0000","last_change":"2024-02-10T06:56:47.054447+0000","last_active":"2024-02-10T06:57:12.045136+0000","last_peered":"2024-02-10T06:57:12.045136+0000","last_clean":"2024-02-10T06:57:12.045136+0000","last_became_active":"2024-02-10T06:56:47.053905+0000","last_became_peered":"2024-02-10T06:56:47.053905+0000","last_unstale":"2024-02-10T06:57:12.045136+0000","last_undegraded":"2024-02-10T06:57:12.045136+0000","last_fullsized":"2024-02-10T06:57:12.045136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:38:25.899911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046058+0000","last_change":"2024-02-10T06:56:51.071744+0000","last_active":"2024-02-10T06:57:12.046058+0000","last_peered":"2024-02-10T06:57:12.046058+0000","last_clean":"2024-02-10T06:57:12.046058+0000","last_became_active":"2024-02-10T06:56:51.071527+0000","last_became_peered":"2024-02-10T06:56:51.071527+0000","last_unstale":"2024-02-10T06:57:12.046058+0000","last_undegraded":"2024-02-10T06:57:12.046058+0000","last_fullsized":"2024-02-10T06:57:12.046058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:39:51.415958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044195+0000","last_change":"2024-02-10T06:56:49.069954+0000","last_active":"2024-02-10T06:57:12.044195+0000","last_peered":"2024-02-10T06:57:12.044195+0000","last_clean":"2024-02-10T06:57:12.044195+0000","last_became_active":"2024-02-10T06:56:49.069591+0000","last_became_peered":"2024-02-10T06:56:49.069591+0000","last_unstale":"2024-02-10T06:57:12.044195+0000","last_undegraded":"2024-02-10T06:57:12.044195+0000","last_fullsized":"2024-02-10T06:57:12.044195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:51:49.551220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048398+0000","last_change":"2024-02-10T06:56:45.033266+0000","last_active":"2024-02-10T06:57:12.048398+0000","last_peered":"2024-02-10T06:57:12.048398+0000","last_clean":"2024-02-10T06:57:12.048398+0000","last_became_active":"2024-02-10T06:56:45.032990+0000","last_became_peered":"2024-02-10T06:56:45.032990+0000","last_unstale":"2024-02-10T06:57:12.048398+0000","last_undegraded":"2024-02-10T06:57:12.048398+0000","last_fullsized":"2024-02-10T06:57:12.048398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:43:41.083512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043944+0000","last_change":"2024-02-10T06:56:47.050969+0000","last_active":"2024-02-10T06:57:12.043944+0000","last_peered":"2024-02-10T06:57:12.043944+0000","last_clean":"2024-02-10T06:57:12.043944+0000","last_became_active":"2024-02-10T06:56:47.050612+0000","last_became_peered":"2024-02-10T06:56:47.050612+0000","last_unstale":"2024-02-10T06:57:12.043944+0000","last_undegraded":"2024-02-10T06:57:12.043944+0000","last_fullsized":"2024-02-10T06:57:12.043944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:39:05.857754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043932+0000","last_change":"2024-02-10T06:56:51.074659+0000","last_active":"2024-02-10T06:57:12.043932+0000","last_peered":"2024-02-10T06:57:12.043932+0000","last_clean":"2024-02-10T06:57:12.043932+0000","last_became_active":"2024-02-10T06:56:51.073942+0000","last_became_peered":"2024-02-10T06:56:51.073942+0000","last_unstale":"2024-02-10T06:57:12.043932+0000","last_undegraded":"2024-02-10T06:57:12.043932+0000","last_fullsized":"2024-02-10T06:57:12.043932+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:32:34.464767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042959+0000","last_change":"2024-02-10T06:56:49.070861+0000","last_active":"2024-02-10T06:57:12.042959+0000","last_peered":"2024-02-10T06:57:12.042959+0000","last_clean":"2024-02-10T06:57:12.042959+0000","last_became_active":"2024-02-10T06:56:49.070490+0000","last_became_peered":"2024-02-10T06:56:49.070490+0000","last_unstale":"2024-02-10T06:57:12.042959+0000","last_undegraded":"2024-02-10T06:57:12.042959+0000","last_fullsized":"2024-02-10T06:57:12.042959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:45:26.403395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047086+0000","last_change":"2024-02-10T06:56:45.037080+0000","last_active":"2024-02-10T06:57:12.047086+0000","last_peered":"2024-02-10T06:57:12.047086+0000","last_clean":"2024-02-10T06:57:12.047086+0000","last_became_active":"2024-02-10T06:56:45.036807+0000","last_became_peered":"2024-02-10T06:56:45.036807+0000","last_unstale":"2024-02-10T06:57:12.047086+0000","last_undegraded":"2024-02-10T06:57:12.047086+0000","last_fullsized":"2024-02-10T06:57:12.047086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:26:30.246278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048298+0000","last_change":"2024-02-10T06:56:47.050212+0000","last_active":"2024-02-10T06:57:12.048298+0000","last_peered":"2024-02-10T06:57:12.048298+0000","last_clean":"2024-02-10T06:57:12.048298+0000","last_became_active":"2024-02-10T06:56:47.049420+0000","last_became_peered":"2024-02-10T06:56:47.049420+0000","last_unstale":"2024-02-10T06:57:12.048298+0000","last_undegraded":"2024-02-10T06:57:12.048298+0000","last_fullsized":"2024-02-10T06:57:12.048298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:09:42.629996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043405+0000","last_change":"2024-02-10T06:56:51.073635+0000","last_active":"2024-02-10T06:57:12.043405+0000","last_peered":"2024-02-10T06:57:12.043405+0000","last_clean":"2024-02-10T06:57:12.043405+0000","last_became_active":"2024-02-10T06:56:51.073347+0000","last_became_peered":"2024-02-10T06:56:51.073347+0000","last_unstale":"2024-02-10T06:57:12.043405+0000","last_undegraded":"2024-02-10T06:57:12.043405+0000","last_fullsized":"2024-02-10T06:57:12.043405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:35:23.707885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047430+0000","last_change":"2024-02-10T06:56:49.074696+0000","last_active":"2024-02-10T06:57:12.047430+0000","last_peered":"2024-02-10T06:57:12.047430+0000","last_clean":"2024-02-10T06:57:12.047430+0000","last_became_active":"2024-02-10T06:56:49.074399+0000","last_became_peered":"2024-02-10T06:56:49.074399+0000","last_unstale":"2024-02-10T06:57:12.047430+0000","last_undegraded":"2024-02-10T06:57:12.047430+0000","last_fullsized":"2024-02-10T06:57:12.047430+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:42:56.538809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048151+0000","last_change":"2024-02-10T06:56:45.034006+0000","last_active":"2024-02-10T06:57:12.048151+0000","last_peered":"2024-02-10T06:57:12.048151+0000","last_clean":"2024-02-10T06:57:12.048151+0000","last_became_active":"2024-02-10T06:56:45.033210+0000","last_became_peered":"2024-02-10T06:56:45.033210+0000","last_unstale":"2024-02-10T06:57:12.048151+0000","last_undegraded":"2024-02-10T06:57:12.048151+0000","last_fullsized":"2024-02-10T06:57:12.048151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:47:40.441638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046864+0000","last_change":"2024-02-10T06:56:47.048186+0000","last_active":"2024-02-10T06:57:12.046864+0000","last_peered":"2024-02-10T06:57:12.046864+0000","last_clean":"2024-02-10T06:57:12.046864+0000","last_became_active":"2024-02-10T06:56:47.047893+0000","last_became_peered":"2024-02-10T06:56:47.047893+0000","last_unstale":"2024-02-10T06:57:12.046864+0000","last_undegraded":"2024-02-10T06:57:12.046864+0000","last_fullsized":"2024-02-10T06:57:12.046864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:45.206971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043446+0000","last_change":"2024-02-10T06:56:51.079158+0000","last_active":"2024-02-10T06:57:12.043446+0000","last_peered":"2024-02-10T06:57:12.043446+0000","last_clean":"2024-02-10T06:57:12.043446+0000","last_became_active":"2024-02-10T06:56:51.078281+0000","last_became_peered":"2024-02-10T06:56:51.078281+0000","last_unstale":"2024-02-10T06:57:12.043446+0000","last_undegraded":"2024-02-10T06:57:12.043446+0000","last_fullsized":"2024-02-10T06:57:12.043446+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:07:07.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046496+0000","last_change":"2024-02-10T06:56:49.073402+0000","last_active":"2024-02-10T06:57:12.046496+0000","last_peered":"2024-02-10T06:57:12.046496+0000","last_clean":"2024-02-10T06:57:12.046496+0000","last_became_active":"2024-02-10T06:56:49.072743+0000","last_became_peered":"2024-02-10T06:56:49.072743+0000","last_unstale":"2024-02-10T06:57:12.046496+0000","last_undegraded":"2024-02-10T06:57:12.046496+0000","last_fullsized":"2024-02-10T06:57:12.046496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:41:14.163984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043231+0000","last_change":"2024-02-10T06:56:45.035290+0000","last_active":"2024-02-10T06:57:12.043231+0000","last_peered":"2024-02-10T06:57:12.043231+0000","last_clean":"2024-02-10T06:57:12.043231+0000","last_became_active":"2024-02-10T06:56:45.033441+0000","last_became_peered":"2024-02-10T06:56:45.033441+0000","last_unstale":"2024-02-10T06:57:12.043231+0000","last_undegraded":"2024-02-10T06:57:12.043231+0000","last_fullsized":"2024-02-10T06:57:12.043231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:41:51.364425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047720+0000","last_change":"2024-02-10T06:56:47.049497+0000","last_active":"2024-02-10T06:57:12.047720+0000","last_peered":"2024-02-10T06:57:12.047720+0000","last_clean":"2024-02-10T06:57:12.047720+0000","last_became_active":"2024-02-10T06:56:47.049201+0000","last_became_peered":"2024-02-10T06:56:47.049201+0000","last_unstale":"2024-02-10T06:57:12.047720+0000","last_undegraded":"2024-02-10T06:57:12.047720+0000","last_fullsized":"2024-02-10T06:57:12.047720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:38:48.731216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044030+0000","last_change":"2024-02-10T06:56:51.078289+0000","last_active":"2024-02-10T06:57:12.044030+0000","last_peered":"2024-02-10T06:57:12.044030+0000","last_clean":"2024-02-10T06:57:12.044030+0000","last_became_active":"2024-02-10T06:56:51.077137+0000","last_became_peered":"2024-02-10T06:56:51.077137+0000","last_unstale":"2024-02-10T06:57:12.044030+0000","last_undegraded":"2024-02-10T06:57:12.044030+0000","last_fullsized":"2024-02-10T06:57:12.044030+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:14:22.875120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081631+0000","last_change":"2024-02-10T06:56:49.075441+0000","last_active":"2024-02-10T06:57:20.081631+0000","last_peered":"2024-02-10T06:57:20.081631+0000","last_clean":"2024-02-10T06:57:20.081631+0000","last_became_active":"2024-02-10T06:56:49.074598+0000","last_became_peered":"2024-02-10T06:56:49.074598+0000","last_unstale":"2024-02-10T06:57:20.081631+0000","last_undegraded":"2024-02-10T06:57:20.081631+0000","last_fullsized":"2024-02-10T06:57:20.081631+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:05:12.345515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042707+0000","last_change":"2024-02-10T06:56:45.034078+0000","last_active":"2024-02-10T06:57:12.042707+0000","last_peered":"2024-02-10T06:57:12.042707+0000","last_clean":"2024-02-10T06:57:12.042707+0000","last_became_active":"2024-02-10T06:56:45.033114+0000","last_became_peered":"2024-02-10T06:56:45.033114+0000","last_unstale":"2024-02-10T06:57:12.042707+0000","last_undegraded":"2024-02-10T06:57:12.042707+0000","last_fullsized":"2024-02-10T06:57:12.042707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:35:04.839591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041451+0000","last_change":"2024-02-10T06:56:47.056475+0000","last_active":"2024-02-10T06:57:12.041451+0000","last_peered":"2024-02-10T06:57:12.041451+0000","last_clean":"2024-02-10T06:57:12.041451+0000","last_became_active":"2024-02-10T06:56:47.055721+0000","last_became_peered":"2024-02-10T06:56:47.055721+0000","last_unstale":"2024-02-10T06:57:12.041451+0000","last_undegraded":"2024-02-10T06:57:12.041451+0000","last_fullsized":"2024-02-10T06:57:12.041451+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:22:32.550078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043247+0000","last_change":"2024-02-10T06:56:51.075002+0000","last_active":"2024-02-10T06:57:12.043247+0000","last_peered":"2024-02-10T06:57:12.043247+0000","last_clean":"2024-02-10T06:57:12.043247+0000","last_became_active":"2024-02-10T06:56:51.074664+0000","last_became_peered":"2024-02-10T06:56:51.074664+0000","last_unstale":"2024-02-10T06:57:12.043247+0000","last_undegraded":"2024-02-10T06:57:12.043247+0000","last_fullsized":"2024-02-10T06:57:12.043247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:16:26.626873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044192+0000","last_change":"2024-02-10T06:56:49.071174+0000","last_active":"2024-02-10T06:57:12.044192+0000","last_peered":"2024-02-10T06:57:12.044192+0000","last_clean":"2024-02-10T06:57:12.044192+0000","last_became_active":"2024-02-10T06:56:49.070808+0000","last_became_peered":"2024-02-10T06:56:49.070808+0000","last_unstale":"2024-02-10T06:57:12.044192+0000","last_undegraded":"2024-02-10T06:57:12.044192+0000","last_fullsized":"2024-02-10T06:57:12.044192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:55:56.445517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045736+0000","last_change":"2024-02-10T06:56:45.034986+0000","last_active":"2024-02-10T06:57:12.045736+0000","last_peered":"2024-02-10T06:57:12.045736+0000","last_clean":"2024-02-10T06:57:12.045736+0000","last_became_active":"2024-02-10T06:56:45.034381+0000","last_became_peered":"2024-02-10T06:56:45.034381+0000","last_unstale":"2024-02-10T06:57:12.045736+0000","last_undegraded":"2024-02-10T06:57:12.045736+0000","last_fullsized":"2024-02-10T06:57:12.045736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:00:21.240895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041429+0000","last_change":"2024-02-10T06:56:47.053768+0000","last_active":"2024-02-10T06:57:12.041429+0000","last_peered":"2024-02-10T06:57:12.041429+0000","last_clean":"2024-02-10T06:57:12.041429+0000","last_became_active":"2024-02-10T06:56:47.053103+0000","last_became_peered":"2024-02-10T06:56:47.053103+0000","last_unstale":"2024-02-10T06:57:12.041429+0000","last_undegraded":"2024-02-10T06:57:12.041429+0000","last_fullsized":"2024-02-10T06:57:12.041429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:01.243089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042831+0000","last_change":"2024-02-10T06:56:51.075729+0000","last_active":"2024-02-10T06:57:12.042831+0000","last_peered":"2024-02-10T06:57:12.042831+0000","last_clean":"2024-02-10T06:57:12.042831+0000","last_became_active":"2024-02-10T06:56:51.074639+0000","last_became_peered":"2024-02-10T06:56:51.074639+0000","last_unstale":"2024-02-10T06:57:12.042831+0000","last_undegraded":"2024-02-10T06:57:12.042831+0000","last_fullsized":"2024-02-10T06:57:12.042831+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:03:13.614368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041324+0000","last_change":"2024-02-10T06:56:49.072488+0000","last_active":"2024-02-10T06:57:12.041324+0000","last_peered":"2024-02-10T06:57:12.041324+0000","last_clean":"2024-02-10T06:57:12.041324+0000","last_became_active":"2024-02-10T06:56:49.072132+0000","last_became_peered":"2024-02-10T06:56:49.072132+0000","last_unstale":"2024-02-10T06:57:12.041324+0000","last_undegraded":"2024-02-10T06:57:12.041324+0000","last_fullsized":"2024-02-10T06:57:12.041324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:44:36.587420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044675+0000","last_change":"2024-02-10T06:56:45.035297+0000","last_active":"2024-02-10T06:57:12.044675+0000","last_peered":"2024-02-10T06:57:12.044675+0000","last_clean":"2024-02-10T06:57:12.044675+0000","last_became_active":"2024-02-10T06:56:45.034278+0000","last_became_peered":"2024-02-10T06:56:45.034278+0000","last_unstale":"2024-02-10T06:57:12.044675+0000","last_undegraded":"2024-02-10T06:57:12.044675+0000","last_fullsized":"2024-02-10T06:57:12.044675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:42:07.430982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044355+0000","last_change":"2024-02-10T06:56:47.049561+0000","last_active":"2024-02-10T06:57:12.044355+0000","last_peered":"2024-02-10T06:57:12.044355+0000","last_clean":"2024-02-10T06:57:12.044355+0000","last_became_active":"2024-02-10T06:56:47.049066+0000","last_became_peered":"2024-02-10T06:56:47.049066+0000","last_unstale":"2024-02-10T06:57:12.044355+0000","last_undegraded":"2024-02-10T06:57:12.044355+0000","last_fullsized":"2024-02-10T06:57:12.044355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:12:10.348128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042878+0000","last_change":"2024-02-10T06:56:51.073202+0000","last_active":"2024-02-10T06:57:12.042878+0000","last_peered":"2024-02-10T06:57:12.042878+0000","last_clean":"2024-02-10T06:57:12.042878+0000","last_became_active":"2024-02-10T06:56:51.072932+0000","last_became_peered":"2024-02-10T06:56:51.072932+0000","last_unstale":"2024-02-10T06:57:12.042878+0000","last_undegraded":"2024-02-10T06:57:12.042878+0000","last_fullsized":"2024-02-10T06:57:12.042878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:49.820592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041307+0000","last_change":"2024-02-10T06:56:49.073380+0000","last_active":"2024-02-10T06:57:12.041307+0000","last_peered":"2024-02-10T06:57:12.041307+0000","last_clean":"2024-02-10T06:57:12.041307+0000","last_became_active":"2024-02-10T06:56:49.073040+0000","last_became_peered":"2024-02-10T06:56:49.073040+0000","last_unstale":"2024-02-10T06:57:12.041307+0000","last_undegraded":"2024-02-10T06:57:12.041307+0000","last_fullsized":"2024-02-10T06:57:12.041307+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:00:47.530046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043958+0000","last_change":"2024-02-10T06:56:45.034437+0000","last_active":"2024-02-10T06:57:12.043958+0000","last_peered":"2024-02-10T06:57:12.043958+0000","last_clean":"2024-02-10T06:57:12.043958+0000","last_became_active":"2024-02-10T06:56:45.033533+0000","last_became_peered":"2024-02-10T06:56:45.033533+0000","last_unstale":"2024-02-10T06:57:12.043958+0000","last_undegraded":"2024-02-10T06:57:12.043958+0000","last_fullsized":"2024-02-10T06:57:12.043958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:02:53.329742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045245+0000","last_change":"2024-02-10T06:56:47.056106+0000","last_active":"2024-02-10T06:57:12.045245+0000","last_peered":"2024-02-10T06:57:12.045245+0000","last_clean":"2024-02-10T06:57:12.045245+0000","last_became_active":"2024-02-10T06:56:47.055491+0000","last_became_peered":"2024-02-10T06:56:47.055491+0000","last_unstale":"2024-02-10T06:57:12.045245+0000","last_undegraded":"2024-02-10T06:57:12.045245+0000","last_fullsized":"2024-02-10T06:57:12.045245+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:50:28.754835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046213+0000","last_change":"2024-02-10T06:56:51.071852+0000","last_active":"2024-02-10T06:57:12.046213+0000","last_peered":"2024-02-10T06:57:12.046213+0000","last_clean":"2024-02-10T06:57:12.046213+0000","last_became_active":"2024-02-10T06:56:51.071588+0000","last_became_peered":"2024-02-10T06:56:51.071588+0000","last_unstale":"2024-02-10T06:57:12.046213+0000","last_undegraded":"2024-02-10T06:57:12.046213+0000","last_fullsized":"2024-02-10T06:57:12.046213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:58:15.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044341+0000","last_change":"2024-02-10T06:56:49.070001+0000","last_active":"2024-02-10T06:57:12.044341+0000","last_peered":"2024-02-10T06:57:12.044341+0000","last_clean":"2024-02-10T06:57:12.044341+0000","last_became_active":"2024-02-10T06:56:49.069644+0000","last_became_peered":"2024-02-10T06:56:49.069644+0000","last_unstale":"2024-02-10T06:57:12.044341+0000","last_undegraded":"2024-02-10T06:57:12.044341+0000","last_fullsized":"2024-02-10T06:57:12.044341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:07:36.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048232+0000","last_change":"2024-02-10T06:56:45.034337+0000","last_active":"2024-02-10T06:57:12.048232+0000","last_peered":"2024-02-10T06:57:12.048232+0000","last_clean":"2024-02-10T06:57:12.048232+0000","last_became_active":"2024-02-10T06:56:45.033338+0000","last_became_peered":"2024-02-10T06:56:45.033338+0000","last_unstale":"2024-02-10T06:57:12.048232+0000","last_undegraded":"2024-02-10T06:57:12.048232+0000","last_fullsized":"2024-02-10T06:57:12.048232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:35:48.656665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044264+0000","last_change":"2024-02-10T06:56:47.050214+0000","last_active":"2024-02-10T06:57:12.044264+0000","last_peered":"2024-02-10T06:57:12.044264+0000","last_clean":"2024-02-10T06:57:12.044264+0000","last_became_active":"2024-02-10T06:56:47.049638+0000","last_became_peered":"2024-02-10T06:56:47.049638+0000","last_unstale":"2024-02-10T06:57:12.044264+0000","last_undegraded":"2024-02-10T06:57:12.044264+0000","last_fullsized":"2024-02-10T06:57:12.044264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:23:38.912282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046938+0000","last_change":"2024-02-10T06:56:51.073939+0000","last_active":"2024-02-10T06:57:12.046938+0000","last_peered":"2024-02-10T06:57:12.046938+0000","last_clean":"2024-02-10T06:57:12.046938+0000","last_became_active":"2024-02-10T06:56:51.073559+0000","last_became_peered":"2024-02-10T06:56:51.073559+0000","last_unstale":"2024-02-10T06:57:12.046938+0000","last_undegraded":"2024-02-10T06:57:12.046938+0000","last_fullsized":"2024-02-10T06:57:12.046938+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:34:18.477467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041676+0000","last_change":"2024-02-10T06:56:49.072681+0000","last_active":"2024-02-10T06:57:12.041676+0000","last_peered":"2024-02-10T06:57:12.041676+0000","last_clean":"2024-02-10T06:57:12.041676+0000","last_became_active":"2024-02-10T06:56:49.072108+0000","last_became_peered":"2024-02-10T06:56:49.072108+0000","last_unstale":"2024-02-10T06:57:12.041676+0000","last_undegraded":"2024-02-10T06:57:12.041676+0000","last_fullsized":"2024-02-10T06:57:12.041676+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:40.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043723+0000","last_change":"2024-02-10T06:56:45.034462+0000","last_active":"2024-02-10T06:57:12.043723+0000","last_peered":"2024-02-10T06:57:12.043723+0000","last_clean":"2024-02-10T06:57:12.043723+0000","last_became_active":"2024-02-10T06:56:45.034048+0000","last_became_peered":"2024-02-10T06:56:45.034048+0000","last_unstale":"2024-02-10T06:57:12.043723+0000","last_undegraded":"2024-02-10T06:57:12.043723+0000","last_fullsized":"2024-02-10T06:57:12.043723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:57:37.943903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048011+0000","last_change":"2024-02-10T06:56:47.049240+0000","last_active":"2024-02-10T06:57:12.048011+0000","last_peered":"2024-02-10T06:57:12.048011+0000","last_clean":"2024-02-10T06:57:12.048011+0000","last_became_active":"2024-02-10T06:56:47.048831+0000","last_became_peered":"2024-02-10T06:56:47.048831+0000","last_unstale":"2024-02-10T06:57:12.048011+0000","last_undegraded":"2024-02-10T06:57:12.048011+0000","last_fullsized":"2024-02-10T06:57:12.048011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:25:01.236606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043079+0000","last_change":"2024-02-10T06:56:51.079164+0000","last_active":"2024-02-10T06:57:12.043079+0000","last_peered":"2024-02-10T06:57:12.043079+0000","last_clean":"2024-02-10T06:57:12.043079+0000","last_became_active":"2024-02-10T06:56:51.078570+0000","last_became_peered":"2024-02-10T06:56:51.078570+0000","last_unstale":"2024-02-10T06:57:12.043079+0000","last_undegraded":"2024-02-10T06:57:12.043079+0000","last_fullsized":"2024-02-10T06:57:12.043079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:45:09.634461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081580+0000","last_change":"2024-02-10T06:56:49.073436+0000","last_active":"2024-02-10T06:57:20.081580+0000","last_peered":"2024-02-10T06:57:20.081580+0000","last_clean":"2024-02-10T06:57:20.081580+0000","last_became_active":"2024-02-10T06:56:49.073116+0000","last_became_peered":"2024-02-10T06:56:49.073116+0000","last_unstale":"2024-02-10T06:57:20.081580+0000","last_undegraded":"2024-02-10T06:57:20.081580+0000","last_fullsized":"2024-02-10T06:57:20.081580+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:28:47.506201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047597+0000","last_change":"2024-02-10T06:56:45.036956+0000","last_active":"2024-02-10T06:57:12.047597+0000","last_peered":"2024-02-10T06:57:12.047597+0000","last_clean":"2024-02-10T06:57:12.047597+0000","last_became_active":"2024-02-10T06:56:45.036630+0000","last_became_peered":"2024-02-10T06:56:45.036630+0000","last_unstale":"2024-02-10T06:57:12.047597+0000","last_undegraded":"2024-02-10T06:57:12.047597+0000","last_fullsized":"2024-02-10T06:57:12.047597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:30:30.857351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041903+0000","last_change":"2024-02-10T06:56:47.054208+0000","last_active":"2024-02-10T06:57:12.041903+0000","last_peered":"2024-02-10T06:57:12.041903+0000","last_clean":"2024-02-10T06:57:12.041903+0000","last_became_active":"2024-02-10T06:56:47.053850+0000","last_became_peered":"2024-02-10T06:56:47.053850+0000","last_unstale":"2024-02-10T06:57:12.041903+0000","last_undegraded":"2024-02-10T06:57:12.041903+0000","last_fullsized":"2024-02-10T06:57:12.041903+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:36:46.888360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043230+0000","last_change":"2024-02-10T06:56:51.074400+0000","last_active":"2024-02-10T06:57:12.043230+0000","last_peered":"2024-02-10T06:57:12.043230+0000","last_clean":"2024-02-10T06:57:12.043230+0000","last_became_active":"2024-02-10T06:56:51.074008+0000","last_became_peered":"2024-02-10T06:56:51.074008+0000","last_unstale":"2024-02-10T06:57:12.043230+0000","last_undegraded":"2024-02-10T06:57:12.043230+0000","last_fullsized":"2024-02-10T06:57:12.043230+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:40:22.197203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081652+0000","last_change":"2024-02-10T06:56:49.073227+0000","last_active":"2024-02-10T06:57:20.081652+0000","last_peered":"2024-02-10T06:57:20.081652+0000","last_clean":"2024-02-10T06:57:20.081652+0000","last_became_active":"2024-02-10T06:56:49.072724+0000","last_became_peered":"2024-02-10T06:56:49.072724+0000","last_unstale":"2024-02-10T06:57:20.081652+0000","last_undegraded":"2024-02-10T06:57:20.081652+0000","last_fullsized":"2024-02-10T06:57:20.081652+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:36:15.771169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046539+0000","last_change":"2024-02-10T06:56:45.034185+0000","last_active":"2024-02-10T06:57:12.046539+0000","last_peered":"2024-02-10T06:57:12.046539+0000","last_clean":"2024-02-10T06:57:12.046539+0000","last_became_active":"2024-02-10T06:56:45.032902+0000","last_became_peered":"2024-02-10T06:56:45.032902+0000","last_unstale":"2024-02-10T06:57:12.046539+0000","last_undegraded":"2024-02-10T06:57:12.046539+0000","last_fullsized":"2024-02-10T06:57:12.046539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:54:19.259892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044766+0000","last_change":"2024-02-10T06:56:47.049475+0000","last_active":"2024-02-10T06:57:12.044766+0000","last_peered":"2024-02-10T06:57:12.044766+0000","last_clean":"2024-02-10T06:57:12.044766+0000","last_became_active":"2024-02-10T06:56:47.049094+0000","last_became_peered":"2024-02-10T06:56:47.049094+0000","last_unstale":"2024-02-10T06:57:12.044766+0000","last_undegraded":"2024-02-10T06:57:12.044766+0000","last_fullsized":"2024-02-10T06:57:12.044766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:06:21.802406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042901+0000","last_change":"2024-02-10T06:56:51.076766+0000","last_active":"2024-02-10T06:57:12.042901+0000","last_peered":"2024-02-10T06:57:12.042901+0000","last_clean":"2024-02-10T06:57:12.042901+0000","last_became_active":"2024-02-10T06:56:51.076465+0000","last_became_peered":"2024-02-10T06:56:51.076465+0000","last_unstale":"2024-02-10T06:57:12.042901+0000","last_undegraded":"2024-02-10T06:57:12.042901+0000","last_fullsized":"2024-02-10T06:57:12.042901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:32:09.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045053+0000","last_change":"2024-02-10T06:56:49.072787+0000","last_active":"2024-02-10T06:57:12.045053+0000","last_peered":"2024-02-10T06:57:12.045053+0000","last_clean":"2024-02-10T06:57:12.045053+0000","last_became_active":"2024-02-10T06:56:49.072455+0000","last_became_peered":"2024-02-10T06:56:49.072455+0000","last_unstale":"2024-02-10T06:57:12.045053+0000","last_undegraded":"2024-02-10T06:57:12.045053+0000","last_fullsized":"2024-02-10T06:57:12.045053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:35:01.004020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042680+0000","last_change":"2024-02-10T06:56:45.034002+0000","last_active":"2024-02-10T06:57:12.042680+0000","last_peered":"2024-02-10T06:57:12.042680+0000","last_clean":"2024-02-10T06:57:12.042680+0000","last_became_active":"2024-02-10T06:56:45.033477+0000","last_became_peered":"2024-02-10T06:56:45.033477+0000","last_unstale":"2024-02-10T06:57:12.042680+0000","last_undegraded":"2024-02-10T06:57:12.042680+0000","last_fullsized":"2024-02-10T06:57:12.042680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:31:15.913179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042256+0000","last_change":"2024-02-10T06:56:47.052744+0000","last_active":"2024-02-10T06:57:12.042256+0000","last_peered":"2024-02-10T06:57:12.042256+0000","last_clean":"2024-02-10T06:57:12.042256+0000","last_became_active":"2024-02-10T06:56:47.052490+0000","last_became_peered":"2024-02-10T06:56:47.052490+0000","last_unstale":"2024-02-10T06:57:12.042256+0000","last_undegraded":"2024-02-10T06:57:12.042256+0000","last_fullsized":"2024-02-10T06:57:12.042256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:30.385894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044779+0000","last_change":"2024-02-10T06:56:51.078850+0000","last_active":"2024-02-10T06:57:12.044779+0000","last_peered":"2024-02-10T06:57:12.044779+0000","last_clean":"2024-02-10T06:57:12.044779+0000","last_became_active":"2024-02-10T06:56:51.078470+0000","last_became_peered":"2024-02-10T06:56:51.078470+0000","last_unstale":"2024-02-10T06:57:12.044779+0000","last_undegraded":"2024-02-10T06:57:12.044779+0000","last_fullsized":"2024-02-10T06:57:12.044779+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:55:42.177906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044226+0000","last_change":"2024-02-10T06:56:49.071574+0000","last_active":"2024-02-10T06:57:12.044226+0000","last_peered":"2024-02-10T06:57:12.044226+0000","last_clean":"2024-02-10T06:57:12.044226+0000","last_became_active":"2024-02-10T06:56:49.070921+0000","last_became_peered":"2024-02-10T06:56:49.070921+0000","last_unstale":"2024-02-10T06:57:12.044226+0000","last_undegraded":"2024-02-10T06:57:12.044226+0000","last_fullsized":"2024-02-10T06:57:12.044226+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:39:00.669023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045921+0000","last_change":"2024-02-10T06:56:45.037938+0000","last_active":"2024-02-10T06:57:12.045921+0000","last_peered":"2024-02-10T06:57:12.045921+0000","last_clean":"2024-02-10T06:57:12.045921+0000","last_became_active":"2024-02-10T06:56:45.037610+0000","last_became_peered":"2024-02-10T06:56:45.037610+0000","last_unstale":"2024-02-10T06:57:12.045921+0000","last_undegraded":"2024-02-10T06:57:12.045921+0000","last_fullsized":"2024-02-10T06:57:12.045921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:16:35.539036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046147+0000","last_change":"2024-02-10T06:56:47.051543+0000","last_active":"2024-02-10T06:57:12.046147+0000","last_peered":"2024-02-10T06:57:12.046147+0000","last_clean":"2024-02-10T06:57:12.046147+0000","last_became_active":"2024-02-10T06:56:47.049819+0000","last_became_peered":"2024-02-10T06:56:47.049819+0000","last_unstale":"2024-02-10T06:57:12.046147+0000","last_undegraded":"2024-02-10T06:57:12.046147+0000","last_fullsized":"2024-02-10T06:57:12.046147+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:04:16.059516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046248+0000","last_change":"2024-02-10T06:56:51.073447+0000","last_active":"2024-02-10T06:57:12.046248+0000","last_peered":"2024-02-10T06:57:12.046248+0000","last_clean":"2024-02-10T06:57:12.046248+0000","last_became_active":"2024-02-10T06:56:51.073109+0000","last_became_peered":"2024-02-10T06:56:51.073109+0000","last_unstale":"2024-02-10T06:57:12.046248+0000","last_undegraded":"2024-02-10T06:57:12.046248+0000","last_fullsized":"2024-02-10T06:57:12.046248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:18.532905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048586+0000","last_change":"2024-02-10T06:56:45.033206+0000","last_active":"2024-02-10T06:57:12.048586+0000","last_peered":"2024-02-10T06:57:12.048586+0000","last_clean":"2024-02-10T06:57:12.048586+0000","last_became_active":"2024-02-10T06:56:45.032897+0000","last_became_peered":"2024-02-10T06:56:45.032897+0000","last_unstale":"2024-02-10T06:57:12.048586+0000","last_undegraded":"2024-02-10T06:57:12.048586+0000","last_fullsized":"2024-02-10T06:57:12.048586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:48:21.536224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048455+0000","last_change":"2024-02-10T06:56:47.049174+0000","last_active":"2024-02-10T06:57:12.048455+0000","last_peered":"2024-02-10T06:57:12.048455+0000","last_clean":"2024-02-10T06:57:12.048455+0000","last_became_active":"2024-02-10T06:56:47.048489+0000","last_became_peered":"2024-02-10T06:56:47.048489+0000","last_unstale":"2024-02-10T06:57:12.048455+0000","last_undegraded":"2024-02-10T06:57:12.048455+0000","last_fullsized":"2024-02-10T06:57:12.048455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:44:52.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081635+0000","last_change":"2024-02-10T06:56:49.073586+0000","last_active":"2024-02-10T06:57:20.081635+0000","last_peered":"2024-02-10T06:57:20.081635+0000","last_clean":"2024-02-10T06:57:20.081635+0000","last_became_active":"2024-02-10T06:56:49.073240+0000","last_became_peered":"2024-02-10T06:56:49.073240+0000","last_unstale":"2024-02-10T06:57:20.081635+0000","last_undegraded":"2024-02-10T06:57:20.081635+0000","last_fullsized":"2024-02-10T06:57:20.081635+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:34:37.663463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041779+0000","last_change":"2024-02-10T06:56:51.077696+0000","last_active":"2024-02-10T06:57:12.041779+0000","last_peered":"2024-02-10T06:57:12.041779+0000","last_clean":"2024-02-10T06:57:12.041779+0000","last_became_active":"2024-02-10T06:56:51.077149+0000","last_became_peered":"2024-02-10T06:56:51.077149+0000","last_unstale":"2024-02-10T06:57:12.041779+0000","last_undegraded":"2024-02-10T06:57:12.041779+0000","last_fullsized":"2024-02-10T06:57:12.041779+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:57:23.253199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044709+0000","last_change":"2024-02-10T06:56:45.033971+0000","last_active":"2024-02-10T06:57:12.044709+0000","last_peered":"2024-02-10T06:57:12.044709+0000","last_clean":"2024-02-10T06:57:12.044709+0000","last_became_active":"2024-02-10T06:56:45.032736+0000","last_became_peered":"2024-02-10T06:56:45.032736+0000","last_unstale":"2024-02-10T06:57:12.044709+0000","last_undegraded":"2024-02-10T06:57:12.044709+0000","last_fullsized":"2024-02-10T06:57:12.044709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:43:46.130761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081603+0000","last_change":"2024-02-10T06:56:47.049525+0000","last_active":"2024-02-10T06:57:20.081603+0000","last_peered":"2024-02-10T06:57:20.081603+0000","last_clean":"2024-02-10T06:57:20.081603+0000","last_became_active":"2024-02-10T06:56:47.048972+0000","last_became_peered":"2024-02-10T06:56:47.048972+0000","last_unstale":"2024-02-10T06:57:20.081603+0000","last_undegraded":"2024-02-10T06:57:20.081603+0000","last_fullsized":"2024-02-10T06:57:20.081603+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:34:12.582783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046306+0000","last_change":"2024-02-10T06:56:49.073752+0000","last_active":"2024-02-10T06:57:12.046306+0000","last_peered":"2024-02-10T06:57:12.046306+0000","last_clean":"2024-02-10T06:57:12.046306+0000","last_became_active":"2024-02-10T06:56:49.073479+0000","last_became_peered":"2024-02-10T06:56:49.073479+0000","last_unstale":"2024-02-10T06:57:12.046306+0000","last_undegraded":"2024-02-10T06:57:12.046306+0000","last_fullsized":"2024-02-10T06:57:12.046306+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:21:51.392750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043025+0000","last_change":"2024-02-10T06:56:51.078880+0000","last_active":"2024-02-10T06:57:12.043025+0000","last_peered":"2024-02-10T06:57:12.043025+0000","last_clean":"2024-02-10T06:57:12.043025+0000","last_became_active":"2024-02-10T06:56:51.077734+0000","last_became_peered":"2024-02-10T06:56:51.077734+0000","last_unstale":"2024-02-10T06:57:12.043025+0000","last_undegraded":"2024-02-10T06:57:12.043025+0000","last_fullsized":"2024-02-10T06:57:12.043025+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:59:55.320105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043049+0000","last_change":"2024-02-10T06:56:45.033807+0000","last_active":"2024-02-10T06:57:12.043049+0000","last_peered":"2024-02-10T06:57:12.043049+0000","last_clean":"2024-02-10T06:57:12.043049+0000","last_became_active":"2024-02-10T06:56:45.033404+0000","last_became_peered":"2024-02-10T06:56:45.033404+0000","last_unstale":"2024-02-10T06:57:12.043049+0000","last_undegraded":"2024-02-10T06:57:12.043049+0000","last_fullsized":"2024-02-10T06:57:12.043049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:57:17.993329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045370+0000","last_change":"2024-02-10T06:56:47.049852+0000","last_active":"2024-02-10T06:57:12.045370+0000","last_peered":"2024-02-10T06:57:12.045370+0000","last_clean":"2024-02-10T06:57:12.045370+0000","last_became_active":"2024-02-10T06:56:47.049215+0000","last_became_peered":"2024-02-10T06:56:47.049215+0000","last_unstale":"2024-02-10T06:57:12.045370+0000","last_undegraded":"2024-02-10T06:57:12.045370+0000","last_fullsized":"2024-02-10T06:57:12.045370+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:55.220665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043083+0000","last_change":"2024-02-10T06:56:49.071354+0000","last_active":"2024-02-10T06:57:12.043083+0000","last_peered":"2024-02-10T06:57:12.043083+0000","last_clean":"2024-02-10T06:57:12.043083+0000","last_became_active":"2024-02-10T06:56:49.070740+0000","last_became_peered":"2024-02-10T06:56:49.070740+0000","last_unstale":"2024-02-10T06:57:12.043083+0000","last_undegraded":"2024-02-10T06:57:12.043083+0000","last_fullsized":"2024-02-10T06:57:12.043083+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:53:22.503006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044612+0000","last_change":"2024-02-10T06:56:51.079419+0000","last_active":"2024-02-10T06:57:12.044612+0000","last_peered":"2024-02-10T06:57:12.044612+0000","last_clean":"2024-02-10T06:57:12.044612+0000","last_became_active":"2024-02-10T06:56:51.078973+0000","last_became_peered":"2024-02-10T06:56:51.078973+0000","last_unstale":"2024-02-10T06:57:12.044612+0000","last_undegraded":"2024-02-10T06:57:12.044612+0000","last_fullsized":"2024-02-10T06:57:12.044612+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:38.662382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045793+0000","last_change":"2024-02-10T06:56:47.049061+0000","last_active":"2024-02-10T06:57:12.045793+0000","last_peered":"2024-02-10T06:57:12.045793+0000","last_clean":"2024-02-10T06:57:12.045793+0000","last_became_active":"2024-02-10T06:56:47.048524+0000","last_became_peered":"2024-02-10T06:56:47.048524+0000","last_unstale":"2024-02-10T06:57:12.045793+0000","last_undegraded":"2024-02-10T06:57:12.045793+0000","last_fullsized":"2024-02-10T06:57:12.045793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:40:46.642278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041774+0000","last_change":"2024-02-10T06:56:45.034303+0000","last_active":"2024-02-10T06:57:12.041774+0000","last_peered":"2024-02-10T06:57:12.041774+0000","last_clean":"2024-02-10T06:57:12.041774+0000","last_became_active":"2024-02-10T06:56:45.033826+0000","last_became_peered":"2024-02-10T06:56:45.033826+0000","last_unstale":"2024-02-10T06:57:12.041774+0000","last_undegraded":"2024-02-10T06:57:12.041774+0000","last_fullsized":"2024-02-10T06:57:12.041774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:43:50.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045143+0000","last_change":"2024-02-10T06:56:49.066019+0000","last_active":"2024-02-10T06:57:12.045143+0000","last_peered":"2024-02-10T06:57:12.045143+0000","last_clean":"2024-02-10T06:57:12.045143+0000","last_became_active":"2024-02-10T06:56:49.065748+0000","last_became_peered":"2024-02-10T06:56:49.065748+0000","last_unstale":"2024-02-10T06:57:12.045143+0000","last_undegraded":"2024-02-10T06:57:12.045143+0000","last_fullsized":"2024-02-10T06:57:12.045143+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:53.671766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046158+0000","last_change":"2024-02-10T06:56:51.073364+0000","last_active":"2024-02-10T06:57:12.046158+0000","last_peered":"2024-02-10T06:57:12.046158+0000","last_clean":"2024-02-10T06:57:12.046158+0000","last_became_active":"2024-02-10T06:56:51.073049+0000","last_became_peered":"2024-02-10T06:56:51.073049+0000","last_unstale":"2024-02-10T06:57:12.046158+0000","last_undegraded":"2024-02-10T06:57:12.046158+0000","last_fullsized":"2024-02-10T06:57:12.046158+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:51.640546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041766+0000","last_change":"2024-02-10T06:56:47.055873+0000","last_active":"2024-02-10T06:57:12.041766+0000","last_peered":"2024-02-10T06:57:12.041766+0000","last_clean":"2024-02-10T06:57:12.041766+0000","last_became_active":"2024-02-10T06:56:47.055310+0000","last_became_peered":"2024-02-10T06:56:47.055310+0000","last_unstale":"2024-02-10T06:57:12.041766+0000","last_undegraded":"2024-02-10T06:57:12.041766+0000","last_fullsized":"2024-02-10T06:57:12.041766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:14:53.021307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044521+0000","last_change":"2024-02-10T06:56:45.035281+0000","last_active":"2024-02-10T06:57:12.044521+0000","last_peered":"2024-02-10T06:57:12.044521+0000","last_clean":"2024-02-10T06:57:12.044521+0000","last_became_active":"2024-02-10T06:56:45.033879+0000","last_became_peered":"2024-02-10T06:56:45.033879+0000","last_unstale":"2024-02-10T06:57:12.044521+0000","last_undegraded":"2024-02-10T06:57:12.044521+0000","last_fullsized":"2024-02-10T06:57:12.044521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:02:08.934870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:14.137291+0000","last_change":"2024-02-10T06:56:34.206813+0000","last_active":"2024-02-10T06:57:14.137291+0000","last_peered":"2024-02-10T06:57:14.137291+0000","last_clean":"2024-02-10T06:57:14.137291+0000","last_became_active":"2024-02-10T06:56:34.205675+0000","last_became_peered":"2024-02-10T06:56:34.205675+0000","last_unstale":"2024-02-10T06:57:14.137291+0000","last_undegraded":"2024-02-10T06:57:14.137291+0000","last_fullsized":"2024-02-10T06:57:14.137291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:33.184061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:33.184061+0000","last_clean_scrub_stamp":"2024-02-10T06:56:33.184061+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:09:30.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043778+0000","last_change":"2024-02-10T06:56:49.065495+0000","last_active":"2024-02-10T06:57:12.043778+0000","last_peered":"2024-02-10T06:57:12.043778+0000","last_clean":"2024-02-10T06:57:12.043778+0000","last_became_active":"2024-02-10T06:56:49.065054+0000","last_became_peered":"2024-02-10T06:56:49.065054+0000","last_unstale":"2024-02-10T06:57:12.043778+0000","last_undegraded":"2024-02-10T06:57:12.043778+0000","last_fullsized":"2024-02-10T06:57:12.043778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:33:34.778768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043513+0000","last_change":"2024-02-10T06:56:51.075114+0000","last_active":"2024-02-10T06:57:12.043513+0000","last_peered":"2024-02-10T06:57:12.043513+0000","last_clean":"2024-02-10T06:57:12.043513+0000","last_became_active":"2024-02-10T06:56:51.074739+0000","last_became_peered":"2024-02-10T06:56:51.074739+0000","last_unstale":"2024-02-10T06:57:12.043513+0000","last_undegraded":"2024-02-10T06:57:12.043513+0000","last_fullsized":"2024-02-10T06:57:12.043513+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:00:12.520465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042819+0000","last_change":"2024-02-10T06:56:47.054373+0000","last_active":"2024-02-10T06:57:12.042819+0000","last_peered":"2024-02-10T06:57:12.042819+0000","last_clean":"2024-02-10T06:57:12.042819+0000","last_became_active":"2024-02-10T06:56:47.054053+0000","last_became_peered":"2024-02-10T06:56:47.054053+0000","last_unstale":"2024-02-10T06:57:12.042819+0000","last_undegraded":"2024-02-10T06:57:12.042819+0000","last_fullsized":"2024-02-10T06:57:12.042819+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:36:56.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042332+0000","last_change":"2024-02-10T06:56:45.035656+0000","last_active":"2024-02-10T06:57:12.042332+0000","last_peered":"2024-02-10T06:57:12.042332+0000","last_clean":"2024-02-10T06:57:12.042332+0000","last_became_active":"2024-02-10T06:56:45.035319+0000","last_became_peered":"2024-02-10T06:56:45.035319+0000","last_unstale":"2024-02-10T06:57:12.042332+0000","last_undegraded":"2024-02-10T06:57:12.042332+0000","last_fullsized":"2024-02-10T06:57:12.042332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:01:50.686497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047397+0000","last_change":"2024-02-10T06:56:49.074298+0000","last_active":"2024-02-10T06:57:12.047397+0000","last_peered":"2024-02-10T06:57:12.047397+0000","last_clean":"2024-02-10T06:57:12.047397+0000","last_became_active":"2024-02-10T06:56:49.074025+0000","last_became_peered":"2024-02-10T06:56:49.074025+0000","last_unstale":"2024-02-10T06:57:12.047397+0000","last_undegraded":"2024-02-10T06:57:12.047397+0000","last_fullsized":"2024-02-10T06:57:12.047397+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:56:37.469490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043140+0000","last_change":"2024-02-10T06:56:51.074605+0000","last_active":"2024-02-10T06:57:12.043140+0000","last_peered":"2024-02-10T06:57:12.043140+0000","last_clean":"2024-02-10T06:57:12.043140+0000","last_became_active":"2024-02-10T06:56:51.073889+0000","last_became_peered":"2024-02-10T06:56:51.073889+0000","last_unstale":"2024-02-10T06:57:12.043140+0000","last_undegraded":"2024-02-10T06:57:12.043140+0000","last_fullsized":"2024-02-10T06:57:12.043140+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:42:06.058380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048135+0000","last_change":"2024-02-10T06:56:45.034573+0000","last_active":"2024-02-10T06:57:12.048135+0000","last_peered":"2024-02-10T06:57:12.048135+0000","last_clean":"2024-02-10T06:57:12.048135+0000","last_became_active":"2024-02-10T06:56:45.033733+0000","last_became_peered":"2024-02-10T06:56:45.033733+0000","last_unstale":"2024-02-10T06:57:12.048135+0000","last_undegraded":"2024-02-10T06:57:12.048135+0000","last_fullsized":"2024-02-10T06:57:12.048135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:18:09.954936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081473+0000","last_change":"2024-02-10T06:56:47.056560+0000","last_active":"2024-02-10T06:57:20.081473+0000","last_peered":"2024-02-10T06:57:20.081473+0000","last_clean":"2024-02-10T06:57:20.081473+0000","last_became_active":"2024-02-10T06:56:47.055689+0000","last_became_peered":"2024-02-10T06:56:47.055689+0000","last_unstale":"2024-02-10T06:57:20.081473+0000","last_undegraded":"2024-02-10T06:57:20.081473+0000","last_fullsized":"2024-02-10T06:57:20.081473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:12:07.925607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081636+0000","last_change":"2024-02-10T06:56:49.072694+0000","last_active":"2024-02-10T06:57:20.081636+0000","last_peered":"2024-02-10T06:57:20.081636+0000","last_clean":"2024-02-10T06:57:20.081636+0000","last_became_active":"2024-02-10T06:56:49.072366+0000","last_became_peered":"2024-02-10T06:56:49.072366+0000","last_unstale":"2024-02-10T06:57:20.081636+0000","last_undegraded":"2024-02-10T06:57:20.081636+0000","last_fullsized":"2024-02-10T06:57:20.081636+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:50:26.574275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042479+0000","last_change":"2024-02-10T06:56:51.074290+0000","last_active":"2024-02-10T06:57:12.042479+0000","last_peered":"2024-02-10T06:57:12.042479+0000","last_clean":"2024-02-10T06:57:12.042479+0000","last_became_active":"2024-02-10T06:56:51.073900+0000","last_became_peered":"2024-02-10T06:56:51.073900+0000","last_unstale":"2024-02-10T06:57:12.042479+0000","last_undegraded":"2024-02-10T06:57:12.042479+0000","last_fullsized":"2024-02-10T06:57:12.042479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:53:07.511838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044506+0000","last_change":"2024-02-10T06:56:45.034397+0000","last_active":"2024-02-10T06:57:12.044506+0000","last_peered":"2024-02-10T06:57:12.044506+0000","last_clean":"2024-02-10T06:57:12.044506+0000","last_became_active":"2024-02-10T06:56:45.034092+0000","last_became_peered":"2024-02-10T06:56:45.034092+0000","last_unstale":"2024-02-10T06:57:12.044506+0000","last_undegraded":"2024-02-10T06:57:12.044506+0000","last_fullsized":"2024-02-10T06:57:12.044506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:03:10.662061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045481+0000","last_change":"2024-02-10T06:56:47.056469+0000","last_active":"2024-02-10T06:57:12.045481+0000","last_peered":"2024-02-10T06:57:12.045481+0000","last_clean":"2024-02-10T06:57:12.045481+0000","last_became_active":"2024-02-10T06:56:47.055821+0000","last_became_peered":"2024-02-10T06:56:47.055821+0000","last_unstale":"2024-02-10T06:57:12.045481+0000","last_undegraded":"2024-02-10T06:57:12.045481+0000","last_fullsized":"2024-02-10T06:57:12.045481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:25:32.299560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081407+0000","last_change":"2024-02-10T06:56:49.073132+0000","last_active":"2024-02-10T06:57:20.081407+0000","last_peered":"2024-02-10T06:57:20.081407+0000","last_clean":"2024-02-10T06:57:20.081407+0000","last_became_active":"2024-02-10T06:56:49.072821+0000","last_became_peered":"2024-02-10T06:56:49.072821+0000","last_unstale":"2024-02-10T06:57:20.081407+0000","last_undegraded":"2024-02-10T06:57:20.081407+0000","last_fullsized":"2024-02-10T06:57:20.081407+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:03:59.935170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046150+0000","last_change":"2024-02-10T06:56:51.073446+0000","last_active":"2024-02-10T06:57:12.046150+0000","last_peered":"2024-02-10T06:57:12.046150+0000","last_clean":"2024-02-10T06:57:12.046150+0000","last_became_active":"2024-02-10T06:56:51.073097+0000","last_became_peered":"2024-02-10T06:56:51.073097+0000","last_unstale":"2024-02-10T06:57:12.046150+0000","last_undegraded":"2024-02-10T06:57:12.046150+0000","last_fullsized":"2024-02-10T06:57:12.046150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:18:35.460329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044482+0000","last_change":"2024-02-10T06:56:45.033653+0000","last_active":"2024-02-10T06:57:12.044482+0000","last_peered":"2024-02-10T06:57:12.044482+0000","last_clean":"2024-02-10T06:57:12.044482+0000","last_became_active":"2024-02-10T06:56:45.033278+0000","last_became_peered":"2024-02-10T06:56:45.033278+0000","last_unstale":"2024-02-10T06:57:12.044482+0000","last_undegraded":"2024-02-10T06:57:12.044482+0000","last_fullsized":"2024-02-10T06:57:12.044482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:14:56.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044280+0000","last_change":"2024-02-10T06:56:47.055445+0000","last_active":"2024-02-10T06:57:12.044280+0000","last_peered":"2024-02-10T06:57:12.044280+0000","last_clean":"2024-02-10T06:57:12.044280+0000","last_became_active":"2024-02-10T06:56:47.055124+0000","last_became_peered":"2024-02-10T06:56:47.055124+0000","last_unstale":"2024-02-10T06:57:12.044280+0000","last_undegraded":"2024-02-10T06:57:12.044280+0000","last_fullsized":"2024-02-10T06:57:12.044280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:42:13.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045878+0000","last_change":"2024-02-10T06:56:49.073535+0000","last_active":"2024-02-10T06:57:12.045878+0000","last_peered":"2024-02-10T06:57:12.045878+0000","last_clean":"2024-02-10T06:57:12.045878+0000","last_became_active":"2024-02-10T06:56:49.073180+0000","last_became_peered":"2024-02-10T06:56:49.073180+0000","last_unstale":"2024-02-10T06:57:12.045878+0000","last_undegraded":"2024-02-10T06:57:12.045878+0000","last_fullsized":"2024-02-10T06:57:12.045878+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:05:48.646235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043343+0000","last_change":"2024-02-10T06:56:51.076745+0000","last_active":"2024-02-10T06:57:12.043343+0000","last_peered":"2024-02-10T06:57:12.043343+0000","last_clean":"2024-02-10T06:57:12.043343+0000","last_became_active":"2024-02-10T06:56:51.076514+0000","last_became_peered":"2024-02-10T06:56:51.076514+0000","last_unstale":"2024-02-10T06:57:12.043343+0000","last_undegraded":"2024-02-10T06:57:12.043343+0000","last_fullsized":"2024-02-10T06:57:12.043343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:11:34.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081523+0000","last_change":"2024-02-10T06:56:49.065784+0000","last_active":"2024-02-10T06:57:20.081523+0000","last_peered":"2024-02-10T06:57:20.081523+0000","last_clean":"2024-02-10T06:57:20.081523+0000","last_became_active":"2024-02-10T06:56:49.065397+0000","last_became_peered":"2024-02-10T06:56:49.065397+0000","last_unstale":"2024-02-10T06:57:20.081523+0000","last_undegraded":"2024-02-10T06:57:20.081523+0000","last_fullsized":"2024-02-10T06:57:20.081523+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:13:05.942893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042678+0000","last_change":"2024-02-10T06:56:45.035055+0000","last_active":"2024-02-10T06:57:12.042678+0000","last_peered":"2024-02-10T06:57:12.042678+0000","last_clean":"2024-02-10T06:57:12.042678+0000","last_became_active":"2024-02-10T06:56:45.033337+0000","last_became_peered":"2024-02-10T06:56:45.033337+0000","last_unstale":"2024-02-10T06:57:12.042678+0000","last_undegraded":"2024-02-10T06:57:12.042678+0000","last_fullsized":"2024-02-10T06:57:12.042678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:41:17.064891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046461+0000","last_change":"2024-02-10T06:56:47.048380+0000","last_active":"2024-02-10T06:57:12.046461+0000","last_peered":"2024-02-10T06:57:12.046461+0000","last_clean":"2024-02-10T06:57:12.046461+0000","last_became_active":"2024-02-10T06:56:47.047807+0000","last_became_peered":"2024-02-10T06:56:47.047807+0000","last_unstale":"2024-02-10T06:57:12.046461+0000","last_undegraded":"2024-02-10T06:57:12.046461+0000","last_fullsized":"2024-02-10T06:57:12.046461+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:33:10.863385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047813+0000","last_change":"2024-02-10T06:56:51.074394+0000","last_active":"2024-02-10T06:57:12.047813+0000","last_peered":"2024-02-10T06:57:12.047813+0000","last_clean":"2024-02-10T06:57:12.047813+0000","last_became_active":"2024-02-10T06:56:51.074082+0000","last_became_peered":"2024-02-10T06:56:51.074082+0000","last_unstale":"2024-02-10T06:57:12.047813+0000","last_undegraded":"2024-02-10T06:57:12.047813+0000","last_fullsized":"2024-02-10T06:57:12.047813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:09:38.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045050+0000","last_change":"2024-02-10T06:56:49.066143+0000","last_active":"2024-02-10T06:57:12.045050+0000","last_peered":"2024-02-10T06:57:12.045050+0000","last_clean":"2024-02-10T06:57:12.045050+0000","last_became_active":"2024-02-10T06:56:49.065822+0000","last_became_peered":"2024-02-10T06:56:49.065822+0000","last_unstale":"2024-02-10T06:57:12.045050+0000","last_undegraded":"2024-02-10T06:57:12.045050+0000","last_fullsized":"2024-02-10T06:57:12.045050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:13:41.226479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041960+0000","last_change":"2024-02-10T06:56:45.034599+0000","last_active":"2024-02-10T06:57:12.041960+0000","last_peered":"2024-02-10T06:57:12.041960+0000","last_clean":"2024-02-10T06:57:12.041960+0000","last_became_active":"2024-02-10T06:56:45.032594+0000","last_became_peered":"2024-02-10T06:56:45.032594+0000","last_unstale":"2024-02-10T06:57:12.041960+0000","last_undegraded":"2024-02-10T06:57:12.041960+0000","last_fullsized":"2024-02-10T06:57:12.041960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:58.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045944+0000","last_change":"2024-02-10T06:56:47.052044+0000","last_active":"2024-02-10T06:57:12.045944+0000","last_peered":"2024-02-10T06:57:12.045944+0000","last_clean":"2024-02-10T06:57:12.045944+0000","last_became_active":"2024-02-10T06:56:47.051538+0000","last_became_peered":"2024-02-10T06:56:47.051538+0000","last_unstale":"2024-02-10T06:57:12.045944+0000","last_undegraded":"2024-02-10T06:57:12.045944+0000","last_fullsized":"2024-02-10T06:57:12.045944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:12:09.329537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047377+0000","last_change":"2024-02-10T06:56:51.074053+0000","last_active":"2024-02-10T06:57:12.047377+0000","last_peered":"2024-02-10T06:57:12.047377+0000","last_clean":"2024-02-10T06:57:12.047377+0000","last_became_active":"2024-02-10T06:56:51.073771+0000","last_became_peered":"2024-02-10T06:56:51.073771+0000","last_unstale":"2024-02-10T06:57:12.047377+0000","last_undegraded":"2024-02-10T06:57:12.047377+0000","last_fullsized":"2024-02-10T06:57:12.047377+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:16:32.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047015+0000","last_change":"2024-02-10T06:56:45.034304+0000","last_active":"2024-02-10T06:57:12.047015+0000","last_peered":"2024-02-10T06:57:12.047015+0000","last_clean":"2024-02-10T06:57:12.047015+0000","last_became_active":"2024-02-10T06:56:45.033627+0000","last_became_peered":"2024-02-10T06:56:45.033627+0000","last_unstale":"2024-02-10T06:57:12.047015+0000","last_undegraded":"2024-02-10T06:57:12.047015+0000","last_fullsized":"2024-02-10T06:57:12.047015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:06:46.448014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042122+0000","last_change":"2024-02-10T06:56:47.054188+0000","last_active":"2024-02-10T06:57:12.042122+0000","last_peered":"2024-02-10T06:57:12.042122+0000","last_clean":"2024-02-10T06:57:12.042122+0000","last_became_active":"2024-02-10T06:56:47.053744+0000","last_became_peered":"2024-02-10T06:56:47.053744+0000","last_unstale":"2024-02-10T06:57:12.042122+0000","last_undegraded":"2024-02-10T06:57:12.042122+0000","last_fullsized":"2024-02-10T06:57:12.042122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:38:23.171411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043515+0000","last_change":"2024-02-10T06:56:49.065914+0000","last_active":"2024-02-10T06:57:12.043515+0000","last_peered":"2024-02-10T06:57:12.043515+0000","last_clean":"2024-02-10T06:57:12.043515+0000","last_became_active":"2024-02-10T06:56:49.065524+0000","last_became_peered":"2024-02-10T06:56:49.065524+0000","last_unstale":"2024-02-10T06:57:12.043515+0000","last_undegraded":"2024-02-10T06:57:12.043515+0000","last_fullsized":"2024-02-10T06:57:12.043515+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:29:14.739985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042417+0000","last_change":"2024-02-10T06:56:51.078882+0000","last_active":"2024-02-10T06:57:12.042417+0000","last_peered":"2024-02-10T06:57:12.042417+0000","last_clean":"2024-02-10T06:57:12.042417+0000","last_became_active":"2024-02-10T06:56:51.077581+0000","last_became_peered":"2024-02-10T06:56:51.077581+0000","last_unstale":"2024-02-10T06:57:12.042417+0000","last_undegraded":"2024-02-10T06:57:12.042417+0000","last_fullsized":"2024-02-10T06:57:12.042417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:09:50.115481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:20.081517+0000","last_change":"2024-02-10T06:56:49.066186+0000","last_active":"2024-02-10T06:57:20.081517+0000","last_peered":"2024-02-10T06:57:20.081517+0000","last_clean":"2024-02-10T06:57:20.081517+0000","last_became_active":"2024-02-10T06:56:49.065864+0000","last_became_peered":"2024-02-10T06:56:49.065864+0000","last_unstale":"2024-02-10T06:57:20.081517+0000","last_undegraded":"2024-02-10T06:57:20.081517+0000","last_fullsized":"2024-02-10T06:57:20.081517+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:13:03.832075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047832+0000","last_change":"2024-02-10T06:56:45.034342+0000","last_active":"2024-02-10T06:57:12.047832+0000","last_peered":"2024-02-10T06:57:12.047832+0000","last_clean":"2024-02-10T06:57:12.047832+0000","last_became_active":"2024-02-10T06:56:45.033601+0000","last_became_peered":"2024-02-10T06:56:45.033601+0000","last_unstale":"2024-02-10T06:57:12.047832+0000","last_undegraded":"2024-02-10T06:57:12.047832+0000","last_fullsized":"2024-02-10T06:57:12.047832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:28.090435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043375+0000","last_change":"2024-02-10T06:56:47.049326+0000","last_active":"2024-02-10T06:57:12.043375+0000","last_peered":"2024-02-10T06:57:12.043375+0000","last_clean":"2024-02-10T06:57:12.043375+0000","last_became_active":"2024-02-10T06:56:47.048841+0000","last_became_peered":"2024-02-10T06:56:47.048841+0000","last_unstale":"2024-02-10T06:57:12.043375+0000","last_undegraded":"2024-02-10T06:57:12.043375+0000","last_fullsized":"2024-02-10T06:57:12.043375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:11:03.763961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047088+0000","last_change":"2024-02-10T06:56:51.073441+0000","last_active":"2024-02-10T06:57:12.047088+0000","last_peered":"2024-02-10T06:57:12.047088+0000","last_clean":"2024-02-10T06:57:12.047088+0000","last_became_active":"2024-02-10T06:56:51.073108+0000","last_became_peered":"2024-02-10T06:56:51.073108+0000","last_unstale":"2024-02-10T06:57:12.047088+0000","last_undegraded":"2024-02-10T06:57:12.047088+0000","last_fullsized":"2024-02-10T06:57:12.047088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:10:48.590841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047665+0000","last_change":"2024-02-10T06:56:45.034694+0000","last_active":"2024-02-10T06:57:12.047665+0000","last_peered":"2024-02-10T06:57:12.047665+0000","last_clean":"2024-02-10T06:57:12.047665+0000","last_became_active":"2024-02-10T06:56:45.034205+0000","last_became_peered":"2024-02-10T06:56:45.034205+0000","last_unstale":"2024-02-10T06:57:12.047665+0000","last_undegraded":"2024-02-10T06:57:12.047665+0000","last_fullsized":"2024-02-10T06:57:12.047665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:48:19.301919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044179+0000","last_change":"2024-02-10T06:56:47.052294+0000","last_active":"2024-02-10T06:57:12.044179+0000","last_peered":"2024-02-10T06:57:12.044179+0000","last_clean":"2024-02-10T06:57:12.044179+0000","last_became_active":"2024-02-10T06:56:47.051956+0000","last_became_peered":"2024-02-10T06:56:47.051956+0000","last_unstale":"2024-02-10T06:57:12.044179+0000","last_undegraded":"2024-02-10T06:57:12.044179+0000","last_fullsized":"2024-02-10T06:57:12.044179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:37:38.445332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042839+0000","last_change":"2024-02-10T06:56:49.065447+0000","last_active":"2024-02-10T06:57:12.042839+0000","last_peered":"2024-02-10T06:57:12.042839+0000","last_clean":"2024-02-10T06:57:12.042839+0000","last_became_active":"2024-02-10T06:56:49.065052+0000","last_became_peered":"2024-02-10T06:56:49.065052+0000","last_unstale":"2024-02-10T06:57:12.042839+0000","last_undegraded":"2024-02-10T06:57:12.042839+0000","last_fullsized":"2024-02-10T06:57:12.042839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:43.610849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041161+0000","last_change":"2024-02-10T06:56:51.077633+0000","last_active":"2024-02-10T06:57:12.041161+0000","last_peered":"2024-02-10T06:57:12.041161+0000","last_clean":"2024-02-10T06:57:12.041161+0000","last_became_active":"2024-02-10T06:56:51.077158+0000","last_became_peered":"2024-02-10T06:56:51.077158+0000","last_unstale":"2024-02-10T06:57:12.041161+0000","last_undegraded":"2024-02-10T06:57:12.041161+0000","last_fullsized":"2024-02-10T06:57:12.041161+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:19:27.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048006+0000","last_change":"2024-02-10T06:56:45.037343+0000","last_active":"2024-02-10T06:57:12.048006+0000","last_peered":"2024-02-10T06:57:12.048006+0000","last_clean":"2024-02-10T06:57:12.048006+0000","last_became_active":"2024-02-10T06:56:45.037072+0000","last_became_peered":"2024-02-10T06:56:45.037072+0000","last_unstale":"2024-02-10T06:57:12.048006+0000","last_undegraded":"2024-02-10T06:57:12.048006+0000","last_fullsized":"2024-02-10T06:57:12.048006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:04:58.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043812+0000","last_change":"2024-02-10T06:56:47.050308+0000","last_active":"2024-02-10T06:57:12.043812+0000","last_peered":"2024-02-10T06:57:12.043812+0000","last_clean":"2024-02-10T06:57:12.043812+0000","last_became_active":"2024-02-10T06:56:47.049811+0000","last_became_peered":"2024-02-10T06:56:47.049811+0000","last_unstale":"2024-02-10T06:57:12.043812+0000","last_undegraded":"2024-02-10T06:57:12.043812+0000","last_fullsized":"2024-02-10T06:57:12.043812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:33:18.649222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041918+0000","last_change":"2024-02-10T06:56:49.071583+0000","last_active":"2024-02-10T06:57:12.041918+0000","last_peered":"2024-02-10T06:57:12.041918+0000","last_clean":"2024-02-10T06:57:12.041918+0000","last_became_active":"2024-02-10T06:56:49.071002+0000","last_became_peered":"2024-02-10T06:56:49.071002+0000","last_unstale":"2024-02-10T06:57:12.041918+0000","last_undegraded":"2024-02-10T06:57:12.041918+0000","last_fullsized":"2024-02-10T06:57:12.041918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:39:05.925492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042362+0000","last_change":"2024-02-10T06:56:51.074826+0000","last_active":"2024-02-10T06:57:12.042362+0000","last_peered":"2024-02-10T06:57:12.042362+0000","last_clean":"2024-02-10T06:57:12.042362+0000","last_became_active":"2024-02-10T06:56:51.074474+0000","last_became_peered":"2024-02-10T06:56:51.074474+0000","last_unstale":"2024-02-10T06:57:12.042362+0000","last_undegraded":"2024-02-10T06:57:12.042362+0000","last_fullsized":"2024-02-10T06:57:12.042362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:09:10.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044406+0000","last_change":"2024-02-10T06:56:45.033915+0000","last_active":"2024-02-10T06:57:12.044406+0000","last_peered":"2024-02-10T06:57:12.044406+0000","last_clean":"2024-02-10T06:57:12.044406+0000","last_became_active":"2024-02-10T06:56:45.032773+0000","last_became_peered":"2024-02-10T06:56:45.032773+0000","last_unstale":"2024-02-10T06:57:12.044406+0000","last_undegraded":"2024-02-10T06:57:12.044406+0000","last_fullsized":"2024-02-10T06:57:12.044406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:29:30.600547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044546+0000","last_change":"2024-02-10T06:56:47.051281+0000","last_active":"2024-02-10T06:57:12.044546+0000","last_peered":"2024-02-10T06:57:12.044546+0000","last_clean":"2024-02-10T06:57:12.044546+0000","last_became_active":"2024-02-10T06:56:47.050712+0000","last_became_peered":"2024-02-10T06:56:47.050712+0000","last_unstale":"2024-02-10T06:57:12.044546+0000","last_undegraded":"2024-02-10T06:57:12.044546+0000","last_fullsized":"2024-02-10T06:57:12.044546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:08:46.266344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046333+0000","last_change":"2024-02-10T06:56:49.075132+0000","last_active":"2024-02-10T06:57:12.046333+0000","last_peered":"2024-02-10T06:57:12.046333+0000","last_clean":"2024-02-10T06:57:12.046333+0000","last_became_active":"2024-02-10T06:56:49.074852+0000","last_became_peered":"2024-02-10T06:56:49.074852+0000","last_unstale":"2024-02-10T06:57:12.046333+0000","last_undegraded":"2024-02-10T06:57:12.046333+0000","last_fullsized":"2024-02-10T06:57:12.046333+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:31:31.579312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 06:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T06:57:33.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T06:57:33.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:33 smithi136 ceph-mon[29743]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:33.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:33 smithi136 ceph-mon[29743]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:33.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:33 smithi189 ceph-mon[30417]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:33.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:33 smithi189 ceph-mon[30417]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:34.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:33 smithi076 ceph-mon[25700]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:34.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:33 smithi076 ceph-mon[25700]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:35.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:35 smithi189 ceph-mon[30417]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:35.940 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:35 smithi076 ceph-mon[25700]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:35.940 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:35 smithi076 podman[58123]: 2024-02-10 06:57:35.749249257 +0000 UTC m=+0.214924426 container create f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:36.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:35 smithi136 ceph-mon[29743]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 podman[58123]: 2024-02-10 06:57:36.068939057 +0000 UTC m=+0.534614226 container init f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 podman[58123]: 2024-02-10 06:57:36.074324587 +0000 UTC m=+0.539999758 container start f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.086Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.086Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.087Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.76 port=9094 2024-02-10T06:57:36.357 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.088Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.110Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.111Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.113Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:36.113Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 bash[58123]: f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 2024-02-10T06:57:36.358 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:36 smithi076 systemd[1]: Started Ceph alertmanager.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE Bus STOPPING 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE Bus STOPPED 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE Bus STARTING 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE Serving on http://:::9283 2024-02-10T06:57:36.613 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:57:36] ENGINE Bus STARTED 2024-02-10T06:57:36.922 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:36.924 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-02-10T06:57:37.228 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:36 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:57:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: Deploying daemon grafana.a on smithi136 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:37.535 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:37 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: Deploying daemon grafana.a on smithi136 2024-02-10T06:57:37.636 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:37.637 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:37 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.656 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-02-10T06:57:36.057061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96616,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126552,"statfs":{"total":287972524032,"available":287873589248,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003068"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043001+0000","last_change":"2024-02-10T06:56:51.075264+0000","last_active":"2024-02-10T06:57:12.043001+0000","last_peered":"2024-02-10T06:57:12.043001+0000","last_clean":"2024-02-10T06:57:12.043001+0000","last_became_active":"2024-02-10T06:56:51.074593+0000","last_became_peered":"2024-02-10T06:56:51.074593+0000","last_unstale":"2024-02-10T06:57:12.043001+0000","last_undegraded":"2024-02-10T06:57:12.043001+0000","last_fullsized":"2024-02-10T06:57:12.043001+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:04:11.432491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045667+0000","last_change":"2024-02-10T06:56:49.072725+0000","last_active":"2024-02-10T06:57:12.045667+0000","last_peered":"2024-02-10T06:57:12.045667+0000","last_clean":"2024-02-10T06:57:12.045667+0000","last_became_active":"2024-02-10T06:56:49.072454+0000","last_became_peered":"2024-02-10T06:56:49.072454+0000","last_unstale":"2024-02-10T06:57:12.045667+0000","last_undegraded":"2024-02-10T06:57:12.045667+0000","last_fullsized":"2024-02-10T06:57:12.045667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:19:09.392591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045389+0000","last_change":"2024-02-10T06:56:45.034172+0000","last_active":"2024-02-10T06:57:12.045389+0000","last_peered":"2024-02-10T06:57:12.045389+0000","last_clean":"2024-02-10T06:57:12.045389+0000","last_became_active":"2024-02-10T06:56:45.033396+0000","last_became_peered":"2024-02-10T06:56:45.033396+0000","last_unstale":"2024-02-10T06:57:12.045389+0000","last_undegraded":"2024-02-10T06:57:12.045389+0000","last_fullsized":"2024-02-10T06:57:12.045389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:52.432749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044316+0000","last_change":"2024-02-10T06:56:47.055864+0000","last_active":"2024-02-10T06:57:12.044316+0000","last_peered":"2024-02-10T06:57:12.044316+0000","last_clean":"2024-02-10T06:57:12.044316+0000","last_became_active":"2024-02-10T06:56:47.055532+0000","last_became_peered":"2024-02-10T06:56:47.055532+0000","last_unstale":"2024-02-10T06:57:12.044316+0000","last_undegraded":"2024-02-10T06:57:12.044316+0000","last_fullsized":"2024-02-10T06:57:12.044316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:06:28.347666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041863+0000","last_change":"2024-02-10T06:56:51.077080+0000","last_active":"2024-02-10T06:57:12.041863+0000","last_peered":"2024-02-10T06:57:12.041863+0000","last_clean":"2024-02-10T06:57:12.041863+0000","last_became_active":"2024-02-10T06:56:51.076740+0000","last_became_peered":"2024-02-10T06:56:51.076740+0000","last_unstale":"2024-02-10T06:57:12.041863+0000","last_undegraded":"2024-02-10T06:57:12.041863+0000","last_fullsized":"2024-02-10T06:57:12.041863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:01:13.462733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042561+0000","last_change":"2024-02-10T06:56:49.071291+0000","last_active":"2024-02-10T06:57:12.042561+0000","last_peered":"2024-02-10T06:57:12.042561+0000","last_clean":"2024-02-10T06:57:12.042561+0000","last_became_active":"2024-02-10T06:56:49.070999+0000","last_became_peered":"2024-02-10T06:56:49.070999+0000","last_unstale":"2024-02-10T06:57:12.042561+0000","last_undegraded":"2024-02-10T06:57:12.042561+0000","last_fullsized":"2024-02-10T06:57:12.042561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:55:50.222415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044493+0000","last_change":"2024-02-10T06:56:45.034169+0000","last_active":"2024-02-10T06:57:12.044493+0000","last_peered":"2024-02-10T06:57:12.044493+0000","last_clean":"2024-02-10T06:57:12.044493+0000","last_became_active":"2024-02-10T06:56:45.033138+0000","last_became_peered":"2024-02-10T06:56:45.033138+0000","last_unstale":"2024-02-10T06:57:12.044493+0000","last_undegraded":"2024-02-10T06:57:12.044493+0000","last_fullsized":"2024-02-10T06:57:12.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:47:59.506891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045136+0000","last_change":"2024-02-10T06:56:47.054447+0000","last_active":"2024-02-10T06:57:12.045136+0000","last_peered":"2024-02-10T06:57:12.045136+0000","last_clean":"2024-02-10T06:57:12.045136+0000","last_became_active":"2024-02-10T06:56:47.053905+0000","last_became_peered":"2024-02-10T06:56:47.053905+0000","last_unstale":"2024-02-10T06:57:12.045136+0000","last_undegraded":"2024-02-10T06:57:12.045136+0000","last_fullsized":"2024-02-10T06:57:12.045136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:38:25.899911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046058+0000","last_change":"2024-02-10T06:56:51.071744+0000","last_active":"2024-02-10T06:57:12.046058+0000","last_peered":"2024-02-10T06:57:12.046058+0000","last_clean":"2024-02-10T06:57:12.046058+0000","last_became_active":"2024-02-10T06:56:51.071527+0000","last_became_peered":"2024-02-10T06:56:51.071527+0000","last_unstale":"2024-02-10T06:57:12.046058+0000","last_undegraded":"2024-02-10T06:57:12.046058+0000","last_fullsized":"2024-02-10T06:57:12.046058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:39:51.415958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044195+0000","last_change":"2024-02-10T06:56:49.069954+0000","last_active":"2024-02-10T06:57:12.044195+0000","last_peered":"2024-02-10T06:57:12.044195+0000","last_clean":"2024-02-10T06:57:12.044195+0000","last_became_active":"2024-02-10T06:56:49.069591+0000","last_became_peered":"2024-02-10T06:56:49.069591+0000","last_unstale":"2024-02-10T06:57:12.044195+0000","last_undegraded":"2024-02-10T06:57:12.044195+0000","last_fullsized":"2024-02-10T06:57:12.044195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:51:49.551220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048398+0000","last_change":"2024-02-10T06:56:45.033266+0000","last_active":"2024-02-10T06:57:12.048398+0000","last_peered":"2024-02-10T06:57:12.048398+0000","last_clean":"2024-02-10T06:57:12.048398+0000","last_became_active":"2024-02-10T06:56:45.032990+0000","last_became_peered":"2024-02-10T06:56:45.032990+0000","last_unstale":"2024-02-10T06:57:12.048398+0000","last_undegraded":"2024-02-10T06:57:12.048398+0000","last_fullsized":"2024-02-10T06:57:12.048398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:43:41.083512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043944+0000","last_change":"2024-02-10T06:56:47.050969+0000","last_active":"2024-02-10T06:57:12.043944+0000","last_peered":"2024-02-10T06:57:12.043944+0000","last_clean":"2024-02-10T06:57:12.043944+0000","last_became_active":"2024-02-10T06:56:47.050612+0000","last_became_peered":"2024-02-10T06:56:47.050612+0000","last_unstale":"2024-02-10T06:57:12.043944+0000","last_undegraded":"2024-02-10T06:57:12.043944+0000","last_fullsized":"2024-02-10T06:57:12.043944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:39:05.857754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043932+0000","last_change":"2024-02-10T06:56:51.074659+0000","last_active":"2024-02-10T06:57:12.043932+0000","last_peered":"2024-02-10T06:57:12.043932+0000","last_clean":"2024-02-10T06:57:12.043932+0000","last_became_active":"2024-02-10T06:56:51.073942+0000","last_became_peered":"2024-02-10T06:56:51.073942+0000","last_unstale":"2024-02-10T06:57:12.043932+0000","last_undegraded":"2024-02-10T06:57:12.043932+0000","last_fullsized":"2024-02-10T06:57:12.043932+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:32:34.464767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042959+0000","last_change":"2024-02-10T06:56:49.070861+0000","last_active":"2024-02-10T06:57:12.042959+0000","last_peered":"2024-02-10T06:57:12.042959+0000","last_clean":"2024-02-10T06:57:12.042959+0000","last_became_active":"2024-02-10T06:56:49.070490+0000","last_became_peered":"2024-02-10T06:56:49.070490+0000","last_unstale":"2024-02-10T06:57:12.042959+0000","last_undegraded":"2024-02-10T06:57:12.042959+0000","last_fullsized":"2024-02-10T06:57:12.042959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:45:26.403395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047086+0000","last_change":"2024-02-10T06:56:45.037080+0000","last_active":"2024-02-10T06:57:12.047086+0000","last_peered":"2024-02-10T06:57:12.047086+0000","last_clean":"2024-02-10T06:57:12.047086+0000","last_became_active":"2024-02-10T06:56:45.036807+0000","last_became_peered":"2024-02-10T06:56:45.036807+0000","last_unstale":"2024-02-10T06:57:12.047086+0000","last_undegraded":"2024-02-10T06:57:12.047086+0000","last_fullsized":"2024-02-10T06:57:12.047086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:26:30.246278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048298+0000","last_change":"2024-02-10T06:56:47.050212+0000","last_active":"2024-02-10T06:57:12.048298+0000","last_peered":"2024-02-10T06:57:12.048298+0000","last_clean":"2024-02-10T06:57:12.048298+0000","last_became_active":"2024-02-10T06:56:47.049420+0000","last_became_peered":"2024-02-10T06:56:47.049420+0000","last_unstale":"2024-02-10T06:57:12.048298+0000","last_undegraded":"2024-02-10T06:57:12.048298+0000","last_fullsized":"2024-02-10T06:57:12.048298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:09:42.629996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043405+0000","last_change":"2024-02-10T06:56:51.073635+0000","last_active":"2024-02-10T06:57:12.043405+0000","last_peered":"2024-02-10T06:57:12.043405+0000","last_clean":"2024-02-10T06:57:12.043405+0000","last_became_active":"2024-02-10T06:56:51.073347+0000","last_became_peered":"2024-02-10T06:56:51.073347+0000","last_unstale":"2024-02-10T06:57:12.043405+0000","last_undegraded":"2024-02-10T06:57:12.043405+0000","last_fullsized":"2024-02-10T06:57:12.043405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:35:23.707885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047430+0000","last_change":"2024-02-10T06:56:49.074696+0000","last_active":"2024-02-10T06:57:12.047430+0000","last_peered":"2024-02-10T06:57:12.047430+0000","last_clean":"2024-02-10T06:57:12.047430+0000","last_became_active":"2024-02-10T06:56:49.074399+0000","last_became_peered":"2024-02-10T06:56:49.074399+0000","last_unstale":"2024-02-10T06:57:12.047430+0000","last_undegraded":"2024-02-10T06:57:12.047430+0000","last_fullsized":"2024-02-10T06:57:12.047430+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:42:56.538809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048151+0000","last_change":"2024-02-10T06:56:45.034006+0000","last_active":"2024-02-10T06:57:12.048151+0000","last_peered":"2024-02-10T06:57:12.048151+0000","last_clean":"2024-02-10T06:57:12.048151+0000","last_became_active":"2024-02-10T06:56:45.033210+0000","last_became_peered":"2024-02-10T06:56:45.033210+0000","last_unstale":"2024-02-10T06:57:12.048151+0000","last_undegraded":"2024-02-10T06:57:12.048151+0000","last_fullsized":"2024-02-10T06:57:12.048151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:47:40.441638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046864+0000","last_change":"2024-02-10T06:56:47.048186+0000","last_active":"2024-02-10T06:57:12.046864+0000","last_peered":"2024-02-10T06:57:12.046864+0000","last_clean":"2024-02-10T06:57:12.046864+0000","last_became_active":"2024-02-10T06:56:47.047893+0000","last_became_peered":"2024-02-10T06:56:47.047893+0000","last_unstale":"2024-02-10T06:57:12.046864+0000","last_undegraded":"2024-02-10T06:57:12.046864+0000","last_fullsized":"2024-02-10T06:57:12.046864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:45.206971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043446+0000","last_change":"2024-02-10T06:56:51.079158+0000","last_active":"2024-02-10T06:57:12.043446+0000","last_peered":"2024-02-10T06:57:12.043446+0000","last_clean":"2024-02-10T06:57:12.043446+0000","last_became_active":"2024-02-10T06:56:51.078281+0000","last_became_peered":"2024-02-10T06:56:51.078281+0000","last_unstale":"2024-02-10T06:57:12.043446+0000","last_undegraded":"2024-02-10T06:57:12.043446+0000","last_fullsized":"2024-02-10T06:57:12.043446+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:07:07.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046496+0000","last_change":"2024-02-10T06:56:49.073402+0000","last_active":"2024-02-10T06:57:12.046496+0000","last_peered":"2024-02-10T06:57:12.046496+0000","last_clean":"2024-02-10T06:57:12.046496+0000","last_became_active":"2024-02-10T06:56:49.072743+0000","last_became_peered":"2024-02-10T06:56:49.072743+0000","last_unstale":"2024-02-10T06:57:12.046496+0000","last_undegraded":"2024-02-10T06:57:12.046496+0000","last_fullsized":"2024-02-10T06:57:12.046496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:41:14.163984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043231+0000","last_change":"2024-02-10T06:56:45.035290+0000","last_active":"2024-02-10T06:57:12.043231+0000","last_peered":"2024-02-10T06:57:12.043231+0000","last_clean":"2024-02-10T06:57:12.043231+0000","last_became_active":"2024-02-10T06:56:45.033441+0000","last_became_peered":"2024-02-10T06:56:45.033441+0000","last_unstale":"2024-02-10T06:57:12.043231+0000","last_undegraded":"2024-02-10T06:57:12.043231+0000","last_fullsized":"2024-02-10T06:57:12.043231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:41:51.364425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047720+0000","last_change":"2024-02-10T06:56:47.049497+0000","last_active":"2024-02-10T06:57:12.047720+0000","last_peered":"2024-02-10T06:57:12.047720+0000","last_clean":"2024-02-10T06:57:12.047720+0000","last_became_active":"2024-02-10T06:56:47.049201+0000","last_became_peered":"2024-02-10T06:56:47.049201+0000","last_unstale":"2024-02-10T06:57:12.047720+0000","last_undegraded":"2024-02-10T06:57:12.047720+0000","last_fullsized":"2024-02-10T06:57:12.047720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:38:48.731216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044030+0000","last_change":"2024-02-10T06:56:51.078289+0000","last_active":"2024-02-10T06:57:12.044030+0000","last_peered":"2024-02-10T06:57:12.044030+0000","last_clean":"2024-02-10T06:57:12.044030+0000","last_became_active":"2024-02-10T06:56:51.077137+0000","last_became_peered":"2024-02-10T06:56:51.077137+0000","last_unstale":"2024-02-10T06:57:12.044030+0000","last_undegraded":"2024-02-10T06:57:12.044030+0000","last_fullsized":"2024-02-10T06:57:12.044030+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:14:22.875120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082623+0000","last_change":"2024-02-10T06:56:49.075441+0000","last_active":"2024-02-10T06:57:25.082623+0000","last_peered":"2024-02-10T06:57:25.082623+0000","last_clean":"2024-02-10T06:57:25.082623+0000","last_became_active":"2024-02-10T06:56:49.074598+0000","last_became_peered":"2024-02-10T06:56:49.074598+0000","last_unstale":"2024-02-10T06:57:25.082623+0000","last_undegraded":"2024-02-10T06:57:25.082623+0000","last_fullsized":"2024-02-10T06:57:25.082623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:05:12.345515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042707+0000","last_change":"2024-02-10T06:56:45.034078+0000","last_active":"2024-02-10T06:57:12.042707+0000","last_peered":"2024-02-10T06:57:12.042707+0000","last_clean":"2024-02-10T06:57:12.042707+0000","last_became_active":"2024-02-10T06:56:45.033114+0000","last_became_peered":"2024-02-10T06:56:45.033114+0000","last_unstale":"2024-02-10T06:57:12.042707+0000","last_undegraded":"2024-02-10T06:57:12.042707+0000","last_fullsized":"2024-02-10T06:57:12.042707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:35:04.839591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041451+0000","last_change":"2024-02-10T06:56:47.056475+0000","last_active":"2024-02-10T06:57:12.041451+0000","last_peered":"2024-02-10T06:57:12.041451+0000","last_clean":"2024-02-10T06:57:12.041451+0000","last_became_active":"2024-02-10T06:56:47.055721+0000","last_became_peered":"2024-02-10T06:56:47.055721+0000","last_unstale":"2024-02-10T06:57:12.041451+0000","last_undegraded":"2024-02-10T06:57:12.041451+0000","last_fullsized":"2024-02-10T06:57:12.041451+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:22:32.550078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043247+0000","last_change":"2024-02-10T06:56:51.075002+0000","last_active":"2024-02-10T06:57:12.043247+0000","last_peered":"2024-02-10T06:57:12.043247+0000","last_clean":"2024-02-10T06:57:12.043247+0000","last_became_active":"2024-02-10T06:56:51.074664+0000","last_became_peered":"2024-02-10T06:56:51.074664+0000","last_unstale":"2024-02-10T06:57:12.043247+0000","last_undegraded":"2024-02-10T06:57:12.043247+0000","last_fullsized":"2024-02-10T06:57:12.043247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:16:26.626873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044192+0000","last_change":"2024-02-10T06:56:49.071174+0000","last_active":"2024-02-10T06:57:12.044192+0000","last_peered":"2024-02-10T06:57:12.044192+0000","last_clean":"2024-02-10T06:57:12.044192+0000","last_became_active":"2024-02-10T06:56:49.070808+0000","last_became_peered":"2024-02-10T06:56:49.070808+0000","last_unstale":"2024-02-10T06:57:12.044192+0000","last_undegraded":"2024-02-10T06:57:12.044192+0000","last_fullsized":"2024-02-10T06:57:12.044192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:55:56.445517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045736+0000","last_change":"2024-02-10T06:56:45.034986+0000","last_active":"2024-02-10T06:57:12.045736+0000","last_peered":"2024-02-10T06:57:12.045736+0000","last_clean":"2024-02-10T06:57:12.045736+0000","last_became_active":"2024-02-10T06:56:45.034381+0000","last_became_peered":"2024-02-10T06:56:45.034381+0000","last_unstale":"2024-02-10T06:57:12.045736+0000","last_undegraded":"2024-02-10T06:57:12.045736+0000","last_fullsized":"2024-02-10T06:57:12.045736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:00:21.240895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041429+0000","last_change":"2024-02-10T06:56:47.053768+0000","last_active":"2024-02-10T06:57:12.041429+0000","last_peered":"2024-02-10T06:57:12.041429+0000","last_clean":"2024-02-10T06:57:12.041429+0000","last_became_active":"2024-02-10T06:56:47.053103+0000","last_became_peered":"2024-02-10T06:56:47.053103+0000","last_unstale":"2024-02-10T06:57:12.041429+0000","last_undegraded":"2024-02-10T06:57:12.041429+0000","last_fullsized":"2024-02-10T06:57:12.041429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:01.243089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042831+0000","last_change":"2024-02-10T06:56:51.075729+0000","last_active":"2024-02-10T06:57:12.042831+0000","last_peered":"2024-02-10T06:57:12.042831+0000","last_clean":"2024-02-10T06:57:12.042831+0000","last_became_active":"2024-02-10T06:56:51.074639+0000","last_became_peered":"2024-02-10T06:56:51.074639+0000","last_unstale":"2024-02-10T06:57:12.042831+0000","last_undegraded":"2024-02-10T06:57:12.042831+0000","last_fullsized":"2024-02-10T06:57:12.042831+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:03:13.614368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041324+0000","last_change":"2024-02-10T06:56:49.072488+0000","last_active":"2024-02-10T06:57:12.041324+0000","last_peered":"2024-02-10T06:57:12.041324+0000","last_clean":"2024-02-10T06:57:12.041324+0000","last_became_active":"2024-02-10T06:56:49.072132+0000","last_became_peered":"2024-02-10T06:56:49.072132+0000","last_unstale":"2024-02-10T06:57:12.041324+0000","last_undegraded":"2024-02-10T06:57:12.041324+0000","last_fullsized":"2024-02-10T06:57:12.041324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:44:36.587420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044675+0000","last_change":"2024-02-10T06:56:45.035297+0000","last_active":"2024-02-10T06:57:12.044675+0000","last_peered":"2024-02-10T06:57:12.044675+0000","last_clean":"2024-02-10T06:57:12.044675+0000","last_became_active":"2024-02-10T06:56:45.034278+0000","last_became_peered":"2024-02-10T06:56:45.034278+0000","last_unstale":"2024-02-10T06:57:12.044675+0000","last_undegraded":"2024-02-10T06:57:12.044675+0000","last_fullsized":"2024-02-10T06:57:12.044675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:42:07.430982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044355+0000","last_change":"2024-02-10T06:56:47.049561+0000","last_active":"2024-02-10T06:57:12.044355+0000","last_peered":"2024-02-10T06:57:12.044355+0000","last_clean":"2024-02-10T06:57:12.044355+0000","last_became_active":"2024-02-10T06:56:47.049066+0000","last_became_peered":"2024-02-10T06:56:47.049066+0000","last_unstale":"2024-02-10T06:57:12.044355+0000","last_undegraded":"2024-02-10T06:57:12.044355+0000","last_fullsized":"2024-02-10T06:57:12.044355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:12:10.348128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042878+0000","last_change":"2024-02-10T06:56:51.073202+0000","last_active":"2024-02-10T06:57:12.042878+0000","last_peered":"2024-02-10T06:57:12.042878+0000","last_clean":"2024-02-10T06:57:12.042878+0000","last_became_active":"2024-02-10T06:56:51.072932+0000","last_became_peered":"2024-02-10T06:56:51.072932+0000","last_unstale":"2024-02-10T06:57:12.042878+0000","last_undegraded":"2024-02-10T06:57:12.042878+0000","last_fullsized":"2024-02-10T06:57:12.042878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:49.820592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041307+0000","last_change":"2024-02-10T06:56:49.073380+0000","last_active":"2024-02-10T06:57:12.041307+0000","last_peered":"2024-02-10T06:57:12.041307+0000","last_clean":"2024-02-10T06:57:12.041307+0000","last_became_active":"2024-02-10T06:56:49.073040+0000","last_became_peered":"2024-02-10T06:56:49.073040+0000","last_unstale":"2024-02-10T06:57:12.041307+0000","last_undegraded":"2024-02-10T06:57:12.041307+0000","last_fullsized":"2024-02-10T06:57:12.041307+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:00:47.530046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043958+0000","last_change":"2024-02-10T06:56:45.034437+0000","last_active":"2024-02-10T06:57:12.043958+0000","last_peered":"2024-02-10T06:57:12.043958+0000","last_clean":"2024-02-10T06:57:12.043958+0000","last_became_active":"2024-02-10T06:56:45.033533+0000","last_became_peered":"2024-02-10T06:56:45.033533+0000","last_unstale":"2024-02-10T06:57:12.043958+0000","last_undegraded":"2024-02-10T06:57:12.043958+0000","last_fullsized":"2024-02-10T06:57:12.043958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:02:53.329742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045245+0000","last_change":"2024-02-10T06:56:47.056106+0000","last_active":"2024-02-10T06:57:12.045245+0000","last_peered":"2024-02-10T06:57:12.045245+0000","last_clean":"2024-02-10T06:57:12.045245+0000","last_became_active":"2024-02-10T06:56:47.055491+0000","last_became_peered":"2024-02-10T06:56:47.055491+0000","last_unstale":"2024-02-10T06:57:12.045245+0000","last_undegraded":"2024-02-10T06:57:12.045245+0000","last_fullsized":"2024-02-10T06:57:12.045245+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:50:28.754835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046213+0000","last_change":"2024-02-10T06:56:51.071852+0000","last_active":"2024-02-10T06:57:12.046213+0000","last_peered":"2024-02-10T06:57:12.046213+0000","last_clean":"2024-02-10T06:57:12.046213+0000","last_became_active":"2024-02-10T06:56:51.071588+0000","last_became_peered":"2024-02-10T06:56:51.071588+0000","last_unstale":"2024-02-10T06:57:12.046213+0000","last_undegraded":"2024-02-10T06:57:12.046213+0000","last_fullsized":"2024-02-10T06:57:12.046213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:58:15.694952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044341+0000","last_change":"2024-02-10T06:56:49.070001+0000","last_active":"2024-02-10T06:57:12.044341+0000","last_peered":"2024-02-10T06:57:12.044341+0000","last_clean":"2024-02-10T06:57:12.044341+0000","last_became_active":"2024-02-10T06:56:49.069644+0000","last_became_peered":"2024-02-10T06:56:49.069644+0000","last_unstale":"2024-02-10T06:57:12.044341+0000","last_undegraded":"2024-02-10T06:57:12.044341+0000","last_fullsized":"2024-02-10T06:57:12.044341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:07:36.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048232+0000","last_change":"2024-02-10T06:56:45.034337+0000","last_active":"2024-02-10T06:57:12.048232+0000","last_peered":"2024-02-10T06:57:12.048232+0000","last_clean":"2024-02-10T06:57:12.048232+0000","last_became_active":"2024-02-10T06:56:45.033338+0000","last_became_peered":"2024-02-10T06:56:45.033338+0000","last_unstale":"2024-02-10T06:57:12.048232+0000","last_undegraded":"2024-02-10T06:57:12.048232+0000","last_fullsized":"2024-02-10T06:57:12.048232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:35:48.656665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044264+0000","last_change":"2024-02-10T06:56:47.050214+0000","last_active":"2024-02-10T06:57:12.044264+0000","last_peered":"2024-02-10T06:57:12.044264+0000","last_clean":"2024-02-10T06:57:12.044264+0000","last_became_active":"2024-02-10T06:56:47.049638+0000","last_became_peered":"2024-02-10T06:56:47.049638+0000","last_unstale":"2024-02-10T06:57:12.044264+0000","last_undegraded":"2024-02-10T06:57:12.044264+0000","last_fullsized":"2024-02-10T06:57:12.044264+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:23:38.912282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046938+0000","last_change":"2024-02-10T06:56:51.073939+0000","last_active":"2024-02-10T06:57:12.046938+0000","last_peered":"2024-02-10T06:57:12.046938+0000","last_clean":"2024-02-10T06:57:12.046938+0000","last_became_active":"2024-02-10T06:56:51.073559+0000","last_became_peered":"2024-02-10T06:56:51.073559+0000","last_unstale":"2024-02-10T06:57:12.046938+0000","last_undegraded":"2024-02-10T06:57:12.046938+0000","last_fullsized":"2024-02-10T06:57:12.046938+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:34:18.477467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041676+0000","last_change":"2024-02-10T06:56:49.072681+0000","last_active":"2024-02-10T06:57:12.041676+0000","last_peered":"2024-02-10T06:57:12.041676+0000","last_clean":"2024-02-10T06:57:12.041676+0000","last_became_active":"2024-02-10T06:56:49.072108+0000","last_became_peered":"2024-02-10T06:56:49.072108+0000","last_unstale":"2024-02-10T06:57:12.041676+0000","last_undegraded":"2024-02-10T06:57:12.041676+0000","last_fullsized":"2024-02-10T06:57:12.041676+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:40.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043723+0000","last_change":"2024-02-10T06:56:45.034462+0000","last_active":"2024-02-10T06:57:12.043723+0000","last_peered":"2024-02-10T06:57:12.043723+0000","last_clean":"2024-02-10T06:57:12.043723+0000","last_became_active":"2024-02-10T06:56:45.034048+0000","last_became_peered":"2024-02-10T06:56:45.034048+0000","last_unstale":"2024-02-10T06:57:12.043723+0000","last_undegraded":"2024-02-10T06:57:12.043723+0000","last_fullsized":"2024-02-10T06:57:12.043723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:57:37.943903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048011+0000","last_change":"2024-02-10T06:56:47.049240+0000","last_active":"2024-02-10T06:57:12.048011+0000","last_peered":"2024-02-10T06:57:12.048011+0000","last_clean":"2024-02-10T06:57:12.048011+0000","last_became_active":"2024-02-10T06:56:47.048831+0000","last_became_peered":"2024-02-10T06:56:47.048831+0000","last_unstale":"2024-02-10T06:57:12.048011+0000","last_undegraded":"2024-02-10T06:57:12.048011+0000","last_fullsized":"2024-02-10T06:57:12.048011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:25:01.236606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043079+0000","last_change":"2024-02-10T06:56:51.079164+0000","last_active":"2024-02-10T06:57:12.043079+0000","last_peered":"2024-02-10T06:57:12.043079+0000","last_clean":"2024-02-10T06:57:12.043079+0000","last_became_active":"2024-02-10T06:56:51.078570+0000","last_became_peered":"2024-02-10T06:56:51.078570+0000","last_unstale":"2024-02-10T06:57:12.043079+0000","last_undegraded":"2024-02-10T06:57:12.043079+0000","last_fullsized":"2024-02-10T06:57:12.043079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:45:09.634461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082655+0000","last_change":"2024-02-10T06:56:49.073436+0000","last_active":"2024-02-10T06:57:25.082655+0000","last_peered":"2024-02-10T06:57:25.082655+0000","last_clean":"2024-02-10T06:57:25.082655+0000","last_became_active":"2024-02-10T06:56:49.073116+0000","last_became_peered":"2024-02-10T06:56:49.073116+0000","last_unstale":"2024-02-10T06:57:25.082655+0000","last_undegraded":"2024-02-10T06:57:25.082655+0000","last_fullsized":"2024-02-10T06:57:25.082655+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:28:47.506201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047597+0000","last_change":"2024-02-10T06:56:45.036956+0000","last_active":"2024-02-10T06:57:12.047597+0000","last_peered":"2024-02-10T06:57:12.047597+0000","last_clean":"2024-02-10T06:57:12.047597+0000","last_became_active":"2024-02-10T06:56:45.036630+0000","last_became_peered":"2024-02-10T06:56:45.036630+0000","last_unstale":"2024-02-10T06:57:12.047597+0000","last_undegraded":"2024-02-10T06:57:12.047597+0000","last_fullsized":"2024-02-10T06:57:12.047597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:30:30.857351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041903+0000","last_change":"2024-02-10T06:56:47.054208+0000","last_active":"2024-02-10T06:57:12.041903+0000","last_peered":"2024-02-10T06:57:12.041903+0000","last_clean":"2024-02-10T06:57:12.041903+0000","last_became_active":"2024-02-10T06:56:47.053850+0000","last_became_peered":"2024-02-10T06:56:47.053850+0000","last_unstale":"2024-02-10T06:57:12.041903+0000","last_undegraded":"2024-02-10T06:57:12.041903+0000","last_fullsized":"2024-02-10T06:57:12.041903+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:36:46.888360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043230+0000","last_change":"2024-02-10T06:56:51.074400+0000","last_active":"2024-02-10T06:57:12.043230+0000","last_peered":"2024-02-10T06:57:12.043230+0000","last_clean":"2024-02-10T06:57:12.043230+0000","last_became_active":"2024-02-10T06:56:51.074008+0000","last_became_peered":"2024-02-10T06:56:51.074008+0000","last_unstale":"2024-02-10T06:57:12.043230+0000","last_undegraded":"2024-02-10T06:57:12.043230+0000","last_fullsized":"2024-02-10T06:57:12.043230+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:40:22.197203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082692+0000","last_change":"2024-02-10T06:56:49.073227+0000","last_active":"2024-02-10T06:57:25.082692+0000","last_peered":"2024-02-10T06:57:25.082692+0000","last_clean":"2024-02-10T06:57:25.082692+0000","last_became_active":"2024-02-10T06:56:49.072724+0000","last_became_peered":"2024-02-10T06:56:49.072724+0000","last_unstale":"2024-02-10T06:57:25.082692+0000","last_undegraded":"2024-02-10T06:57:25.082692+0000","last_fullsized":"2024-02-10T06:57:25.082692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:36:15.771169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046539+0000","last_change":"2024-02-10T06:56:45.034185+0000","last_active":"2024-02-10T06:57:12.046539+0000","last_peered":"2024-02-10T06:57:12.046539+0000","last_clean":"2024-02-10T06:57:12.046539+0000","last_became_active":"2024-02-10T06:56:45.032902+0000","last_became_peered":"2024-02-10T06:56:45.032902+0000","last_unstale":"2024-02-10T06:57:12.046539+0000","last_undegraded":"2024-02-10T06:57:12.046539+0000","last_fullsized":"2024-02-10T06:57:12.046539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:54:19.259892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044766+0000","last_change":"2024-02-10T06:56:47.049475+0000","last_active":"2024-02-10T06:57:12.044766+0000","last_peered":"2024-02-10T06:57:12.044766+0000","last_clean":"2024-02-10T06:57:12.044766+0000","last_became_active":"2024-02-10T06:56:47.049094+0000","last_became_peered":"2024-02-10T06:56:47.049094+0000","last_unstale":"2024-02-10T06:57:12.044766+0000","last_undegraded":"2024-02-10T06:57:12.044766+0000","last_fullsized":"2024-02-10T06:57:12.044766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:06:21.802406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042901+0000","last_change":"2024-02-10T06:56:51.076766+0000","last_active":"2024-02-10T06:57:12.042901+0000","last_peered":"2024-02-10T06:57:12.042901+0000","last_clean":"2024-02-10T06:57:12.042901+0000","last_became_active":"2024-02-10T06:56:51.076465+0000","last_became_peered":"2024-02-10T06:56:51.076465+0000","last_unstale":"2024-02-10T06:57:12.042901+0000","last_undegraded":"2024-02-10T06:57:12.042901+0000","last_fullsized":"2024-02-10T06:57:12.042901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:32:09.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045053+0000","last_change":"2024-02-10T06:56:49.072787+0000","last_active":"2024-02-10T06:57:12.045053+0000","last_peered":"2024-02-10T06:57:12.045053+0000","last_clean":"2024-02-10T06:57:12.045053+0000","last_became_active":"2024-02-10T06:56:49.072455+0000","last_became_peered":"2024-02-10T06:56:49.072455+0000","last_unstale":"2024-02-10T06:57:12.045053+0000","last_undegraded":"2024-02-10T06:57:12.045053+0000","last_fullsized":"2024-02-10T06:57:12.045053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:35:01.004020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042680+0000","last_change":"2024-02-10T06:56:45.034002+0000","last_active":"2024-02-10T06:57:12.042680+0000","last_peered":"2024-02-10T06:57:12.042680+0000","last_clean":"2024-02-10T06:57:12.042680+0000","last_became_active":"2024-02-10T06:56:45.033477+0000","last_became_peered":"2024-02-10T06:56:45.033477+0000","last_unstale":"2024-02-10T06:57:12.042680+0000","last_undegraded":"2024-02-10T06:57:12.042680+0000","last_fullsized":"2024-02-10T06:57:12.042680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:31:15.913179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042256+0000","last_change":"2024-02-10T06:56:47.052744+0000","last_active":"2024-02-10T06:57:12.042256+0000","last_peered":"2024-02-10T06:57:12.042256+0000","last_clean":"2024-02-10T06:57:12.042256+0000","last_became_active":"2024-02-10T06:56:47.052490+0000","last_became_peered":"2024-02-10T06:56:47.052490+0000","last_unstale":"2024-02-10T06:57:12.042256+0000","last_undegraded":"2024-02-10T06:57:12.042256+0000","last_fullsized":"2024-02-10T06:57:12.042256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:30.385894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044779+0000","last_change":"2024-02-10T06:56:51.078850+0000","last_active":"2024-02-10T06:57:12.044779+0000","last_peered":"2024-02-10T06:57:12.044779+0000","last_clean":"2024-02-10T06:57:12.044779+0000","last_became_active":"2024-02-10T06:56:51.078470+0000","last_became_peered":"2024-02-10T06:56:51.078470+0000","last_unstale":"2024-02-10T06:57:12.044779+0000","last_undegraded":"2024-02-10T06:57:12.044779+0000","last_fullsized":"2024-02-10T06:57:12.044779+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:55:42.177906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044226+0000","last_change":"2024-02-10T06:56:49.071574+0000","last_active":"2024-02-10T06:57:12.044226+0000","last_peered":"2024-02-10T06:57:12.044226+0000","last_clean":"2024-02-10T06:57:12.044226+0000","last_became_active":"2024-02-10T06:56:49.070921+0000","last_became_peered":"2024-02-10T06:56:49.070921+0000","last_unstale":"2024-02-10T06:57:12.044226+0000","last_undegraded":"2024-02-10T06:57:12.044226+0000","last_fullsized":"2024-02-10T06:57:12.044226+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:39:00.669023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045921+0000","last_change":"2024-02-10T06:56:45.037938+0000","last_active":"2024-02-10T06:57:12.045921+0000","last_peered":"2024-02-10T06:57:12.045921+0000","last_clean":"2024-02-10T06:57:12.045921+0000","last_became_active":"2024-02-10T06:56:45.037610+0000","last_became_peered":"2024-02-10T06:56:45.037610+0000","last_unstale":"2024-02-10T06:57:12.045921+0000","last_undegraded":"2024-02-10T06:57:12.045921+0000","last_fullsized":"2024-02-10T06:57:12.045921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:16:35.539036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046147+0000","last_change":"2024-02-10T06:56:47.051543+0000","last_active":"2024-02-10T06:57:12.046147+0000","last_peered":"2024-02-10T06:57:12.046147+0000","last_clean":"2024-02-10T06:57:12.046147+0000","last_became_active":"2024-02-10T06:56:47.049819+0000","last_became_peered":"2024-02-10T06:56:47.049819+0000","last_unstale":"2024-02-10T06:57:12.046147+0000","last_undegraded":"2024-02-10T06:57:12.046147+0000","last_fullsized":"2024-02-10T06:57:12.046147+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:04:16.059516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046248+0000","last_change":"2024-02-10T06:56:51.073447+0000","last_active":"2024-02-10T06:57:12.046248+0000","last_peered":"2024-02-10T06:57:12.046248+0000","last_clean":"2024-02-10T06:57:12.046248+0000","last_became_active":"2024-02-10T06:56:51.073109+0000","last_became_peered":"2024-02-10T06:56:51.073109+0000","last_unstale":"2024-02-10T06:57:12.046248+0000","last_undegraded":"2024-02-10T06:57:12.046248+0000","last_fullsized":"2024-02-10T06:57:12.046248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:18.532905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048586+0000","last_change":"2024-02-10T06:56:45.033206+0000","last_active":"2024-02-10T06:57:12.048586+0000","last_peered":"2024-02-10T06:57:12.048586+0000","last_clean":"2024-02-10T06:57:12.048586+0000","last_became_active":"2024-02-10T06:56:45.032897+0000","last_became_peered":"2024-02-10T06:56:45.032897+0000","last_unstale":"2024-02-10T06:57:12.048586+0000","last_undegraded":"2024-02-10T06:57:12.048586+0000","last_fullsized":"2024-02-10T06:57:12.048586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:48:21.536224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048455+0000","last_change":"2024-02-10T06:56:47.049174+0000","last_active":"2024-02-10T06:57:12.048455+0000","last_peered":"2024-02-10T06:57:12.048455+0000","last_clean":"2024-02-10T06:57:12.048455+0000","last_became_active":"2024-02-10T06:56:47.048489+0000","last_became_peered":"2024-02-10T06:56:47.048489+0000","last_unstale":"2024-02-10T06:57:12.048455+0000","last_undegraded":"2024-02-10T06:57:12.048455+0000","last_fullsized":"2024-02-10T06:57:12.048455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:44:52.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082692+0000","last_change":"2024-02-10T06:56:49.073586+0000","last_active":"2024-02-10T06:57:25.082692+0000","last_peered":"2024-02-10T06:57:25.082692+0000","last_clean":"2024-02-10T06:57:25.082692+0000","last_became_active":"2024-02-10T06:56:49.073240+0000","last_became_peered":"2024-02-10T06:56:49.073240+0000","last_unstale":"2024-02-10T06:57:25.082692+0000","last_undegraded":"2024-02-10T06:57:25.082692+0000","last_fullsized":"2024-02-10T06:57:25.082692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:34:37.663463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041779+0000","last_change":"2024-02-10T06:56:51.077696+0000","last_active":"2024-02-10T06:57:12.041779+0000","last_peered":"2024-02-10T06:57:12.041779+0000","last_clean":"2024-02-10T06:57:12.041779+0000","last_became_active":"2024-02-10T06:56:51.077149+0000","last_became_peered":"2024-02-10T06:56:51.077149+0000","last_unstale":"2024-02-10T06:57:12.041779+0000","last_undegraded":"2024-02-10T06:57:12.041779+0000","last_fullsized":"2024-02-10T06:57:12.041779+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:57:23.253199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044709+0000","last_change":"2024-02-10T06:56:45.033971+0000","last_active":"2024-02-10T06:57:12.044709+0000","last_peered":"2024-02-10T06:57:12.044709+0000","last_clean":"2024-02-10T06:57:12.044709+0000","last_became_active":"2024-02-10T06:56:45.032736+0000","last_became_peered":"2024-02-10T06:56:45.032736+0000","last_unstale":"2024-02-10T06:57:12.044709+0000","last_undegraded":"2024-02-10T06:57:12.044709+0000","last_fullsized":"2024-02-10T06:57:12.044709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:43:46.130761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082571+0000","last_change":"2024-02-10T06:56:47.049525+0000","last_active":"2024-02-10T06:57:25.082571+0000","last_peered":"2024-02-10T06:57:25.082571+0000","last_clean":"2024-02-10T06:57:25.082571+0000","last_became_active":"2024-02-10T06:56:47.048972+0000","last_became_peered":"2024-02-10T06:56:47.048972+0000","last_unstale":"2024-02-10T06:57:25.082571+0000","last_undegraded":"2024-02-10T06:57:25.082571+0000","last_fullsized":"2024-02-10T06:57:25.082571+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:34:12.582783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046306+0000","last_change":"2024-02-10T06:56:49.073752+0000","last_active":"2024-02-10T06:57:12.046306+0000","last_peered":"2024-02-10T06:57:12.046306+0000","last_clean":"2024-02-10T06:57:12.046306+0000","last_became_active":"2024-02-10T06:56:49.073479+0000","last_became_peered":"2024-02-10T06:56:49.073479+0000","last_unstale":"2024-02-10T06:57:12.046306+0000","last_undegraded":"2024-02-10T06:57:12.046306+0000","last_fullsized":"2024-02-10T06:57:12.046306+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:21:51.392750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043025+0000","last_change":"2024-02-10T06:56:51.078880+0000","last_active":"2024-02-10T06:57:12.043025+0000","last_peered":"2024-02-10T06:57:12.043025+0000","last_clean":"2024-02-10T06:57:12.043025+0000","last_became_active":"2024-02-10T06:56:51.077734+0000","last_became_peered":"2024-02-10T06:56:51.077734+0000","last_unstale":"2024-02-10T06:57:12.043025+0000","last_undegraded":"2024-02-10T06:57:12.043025+0000","last_fullsized":"2024-02-10T06:57:12.043025+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:59:55.320105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043049+0000","last_change":"2024-02-10T06:56:45.033807+0000","last_active":"2024-02-10T06:57:12.043049+0000","last_peered":"2024-02-10T06:57:12.043049+0000","last_clean":"2024-02-10T06:57:12.043049+0000","last_became_active":"2024-02-10T06:56:45.033404+0000","last_became_peered":"2024-02-10T06:56:45.033404+0000","last_unstale":"2024-02-10T06:57:12.043049+0000","last_undegraded":"2024-02-10T06:57:12.043049+0000","last_fullsized":"2024-02-10T06:57:12.043049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:57:17.993329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045370+0000","last_change":"2024-02-10T06:56:47.049852+0000","last_active":"2024-02-10T06:57:12.045370+0000","last_peered":"2024-02-10T06:57:12.045370+0000","last_clean":"2024-02-10T06:57:12.045370+0000","last_became_active":"2024-02-10T06:56:47.049215+0000","last_became_peered":"2024-02-10T06:56:47.049215+0000","last_unstale":"2024-02-10T06:57:12.045370+0000","last_undegraded":"2024-02-10T06:57:12.045370+0000","last_fullsized":"2024-02-10T06:57:12.045370+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:55.220665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043083+0000","last_change":"2024-02-10T06:56:49.071354+0000","last_active":"2024-02-10T06:57:12.043083+0000","last_peered":"2024-02-10T06:57:12.043083+0000","last_clean":"2024-02-10T06:57:12.043083+0000","last_became_active":"2024-02-10T06:56:49.070740+0000","last_became_peered":"2024-02-10T06:56:49.070740+0000","last_unstale":"2024-02-10T06:57:12.043083+0000","last_undegraded":"2024-02-10T06:57:12.043083+0000","last_fullsized":"2024-02-10T06:57:12.043083+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:53:22.503006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044612+0000","last_change":"2024-02-10T06:56:51.079419+0000","last_active":"2024-02-10T06:57:12.044612+0000","last_peered":"2024-02-10T06:57:12.044612+0000","last_clean":"2024-02-10T06:57:12.044612+0000","last_became_active":"2024-02-10T06:56:51.078973+0000","last_became_peered":"2024-02-10T06:56:51.078973+0000","last_unstale":"2024-02-10T06:57:12.044612+0000","last_undegraded":"2024-02-10T06:57:12.044612+0000","last_fullsized":"2024-02-10T06:57:12.044612+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:38.662382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045793+0000","last_change":"2024-02-10T06:56:47.049061+0000","last_active":"2024-02-10T06:57:12.045793+0000","last_peered":"2024-02-10T06:57:12.045793+0000","last_clean":"2024-02-10T06:57:12.045793+0000","last_became_active":"2024-02-10T06:56:47.048524+0000","last_became_peered":"2024-02-10T06:56:47.048524+0000","last_unstale":"2024-02-10T06:57:12.045793+0000","last_undegraded":"2024-02-10T06:57:12.045793+0000","last_fullsized":"2024-02-10T06:57:12.045793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:40:46.642278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041774+0000","last_change":"2024-02-10T06:56:45.034303+0000","last_active":"2024-02-10T06:57:12.041774+0000","last_peered":"2024-02-10T06:57:12.041774+0000","last_clean":"2024-02-10T06:57:12.041774+0000","last_became_active":"2024-02-10T06:56:45.033826+0000","last_became_peered":"2024-02-10T06:56:45.033826+0000","last_unstale":"2024-02-10T06:57:12.041774+0000","last_undegraded":"2024-02-10T06:57:12.041774+0000","last_fullsized":"2024-02-10T06:57:12.041774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:43:50.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045143+0000","last_change":"2024-02-10T06:56:49.066019+0000","last_active":"2024-02-10T06:57:12.045143+0000","last_peered":"2024-02-10T06:57:12.045143+0000","last_clean":"2024-02-10T06:57:12.045143+0000","last_became_active":"2024-02-10T06:56:49.065748+0000","last_became_peered":"2024-02-10T06:56:49.065748+0000","last_unstale":"2024-02-10T06:57:12.045143+0000","last_undegraded":"2024-02-10T06:57:12.045143+0000","last_fullsized":"2024-02-10T06:57:12.045143+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:53.671766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046158+0000","last_change":"2024-02-10T06:56:51.073364+0000","last_active":"2024-02-10T06:57:12.046158+0000","last_peered":"2024-02-10T06:57:12.046158+0000","last_clean":"2024-02-10T06:57:12.046158+0000","last_became_active":"2024-02-10T06:56:51.073049+0000","last_became_peered":"2024-02-10T06:56:51.073049+0000","last_unstale":"2024-02-10T06:57:12.046158+0000","last_undegraded":"2024-02-10T06:57:12.046158+0000","last_fullsized":"2024-02-10T06:57:12.046158+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:51.640546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041766+0000","last_change":"2024-02-10T06:56:47.055873+0000","last_active":"2024-02-10T06:57:12.041766+0000","last_peered":"2024-02-10T06:57:12.041766+0000","last_clean":"2024-02-10T06:57:12.041766+0000","last_became_active":"2024-02-10T06:56:47.055310+0000","last_became_peered":"2024-02-10T06:56:47.055310+0000","last_unstale":"2024-02-10T06:57:12.041766+0000","last_undegraded":"2024-02-10T06:57:12.041766+0000","last_fullsized":"2024-02-10T06:57:12.041766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:14:53.021307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044521+0000","last_change":"2024-02-10T06:56:45.035281+0000","last_active":"2024-02-10T06:57:12.044521+0000","last_peered":"2024-02-10T06:57:12.044521+0000","last_clean":"2024-02-10T06:57:12.044521+0000","last_became_active":"2024-02-10T06:56:45.033879+0000","last_became_peered":"2024-02-10T06:56:45.033879+0000","last_unstale":"2024-02-10T06:57:12.044521+0000","last_undegraded":"2024-02-10T06:57:12.044521+0000","last_fullsized":"2024-02-10T06:57:12.044521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:02:08.934870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:14.137291+0000","last_change":"2024-02-10T06:56:34.206813+0000","last_active":"2024-02-10T06:57:14.137291+0000","last_peered":"2024-02-10T06:57:14.137291+0000","last_clean":"2024-02-10T06:57:14.137291+0000","last_became_active":"2024-02-10T06:56:34.205675+0000","last_became_peered":"2024-02-10T06:56:34.205675+0000","last_unstale":"2024-02-10T06:57:14.137291+0000","last_undegraded":"2024-02-10T06:57:14.137291+0000","last_fullsized":"2024-02-10T06:57:14.137291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:33.184061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:33.184061+0000","last_clean_scrub_stamp":"2024-02-10T06:56:33.184061+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:09:30.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043778+0000","last_change":"2024-02-10T06:56:49.065495+0000","last_active":"2024-02-10T06:57:12.043778+0000","last_peered":"2024-02-10T06:57:12.043778+0000","last_clean":"2024-02-10T06:57:12.043778+0000","last_became_active":"2024-02-10T06:56:49.065054+0000","last_became_peered":"2024-02-10T06:56:49.065054+0000","last_unstale":"2024-02-10T06:57:12.043778+0000","last_undegraded":"2024-02-10T06:57:12.043778+0000","last_fullsized":"2024-02-10T06:57:12.043778+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:33:34.778768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043513+0000","last_change":"2024-02-10T06:56:51.075114+0000","last_active":"2024-02-10T06:57:12.043513+0000","last_peered":"2024-02-10T06:57:12.043513+0000","last_clean":"2024-02-10T06:57:12.043513+0000","last_became_active":"2024-02-10T06:56:51.074739+0000","last_became_peered":"2024-02-10T06:56:51.074739+0000","last_unstale":"2024-02-10T06:57:12.043513+0000","last_undegraded":"2024-02-10T06:57:12.043513+0000","last_fullsized":"2024-02-10T06:57:12.043513+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:00:12.520465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042819+0000","last_change":"2024-02-10T06:56:47.054373+0000","last_active":"2024-02-10T06:57:12.042819+0000","last_peered":"2024-02-10T06:57:12.042819+0000","last_clean":"2024-02-10T06:57:12.042819+0000","last_became_active":"2024-02-10T06:56:47.054053+0000","last_became_peered":"2024-02-10T06:56:47.054053+0000","last_unstale":"2024-02-10T06:57:12.042819+0000","last_undegraded":"2024-02-10T06:57:12.042819+0000","last_fullsized":"2024-02-10T06:57:12.042819+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:36:56.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042332+0000","last_change":"2024-02-10T06:56:45.035656+0000","last_active":"2024-02-10T06:57:12.042332+0000","last_peered":"2024-02-10T06:57:12.042332+0000","last_clean":"2024-02-10T06:57:12.042332+0000","last_became_active":"2024-02-10T06:56:45.035319+0000","last_became_peered":"2024-02-10T06:56:45.035319+0000","last_unstale":"2024-02-10T06:57:12.042332+0000","last_undegraded":"2024-02-10T06:57:12.042332+0000","last_fullsized":"2024-02-10T06:57:12.042332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:01:50.686497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047397+0000","last_change":"2024-02-10T06:56:49.074298+0000","last_active":"2024-02-10T06:57:12.047397+0000","last_peered":"2024-02-10T06:57:12.047397+0000","last_clean":"2024-02-10T06:57:12.047397+0000","last_became_active":"2024-02-10T06:56:49.074025+0000","last_became_peered":"2024-02-10T06:56:49.074025+0000","last_unstale":"2024-02-10T06:57:12.047397+0000","last_undegraded":"2024-02-10T06:57:12.047397+0000","last_fullsized":"2024-02-10T06:57:12.047397+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:56:37.469490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043140+0000","last_change":"2024-02-10T06:56:51.074605+0000","last_active":"2024-02-10T06:57:12.043140+0000","last_peered":"2024-02-10T06:57:12.043140+0000","last_clean":"2024-02-10T06:57:12.043140+0000","last_became_active":"2024-02-10T06:56:51.073889+0000","last_became_peered":"2024-02-10T06:56:51.073889+0000","last_unstale":"2024-02-10T06:57:12.043140+0000","last_undegraded":"2024-02-10T06:57:12.043140+0000","last_fullsized":"2024-02-10T06:57:12.043140+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:42:06.058380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048135+0000","last_change":"2024-02-10T06:56:45.034573+0000","last_active":"2024-02-10T06:57:12.048135+0000","last_peered":"2024-02-10T06:57:12.048135+0000","last_clean":"2024-02-10T06:57:12.048135+0000","last_became_active":"2024-02-10T06:56:45.033733+0000","last_became_peered":"2024-02-10T06:56:45.033733+0000","last_unstale":"2024-02-10T06:57:12.048135+0000","last_undegraded":"2024-02-10T06:57:12.048135+0000","last_fullsized":"2024-02-10T06:57:12.048135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:18:09.954936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082340+0000","last_change":"2024-02-10T06:56:47.056560+0000","last_active":"2024-02-10T06:57:25.082340+0000","last_peered":"2024-02-10T06:57:25.082340+0000","last_clean":"2024-02-10T06:57:25.082340+0000","last_became_active":"2024-02-10T06:56:47.055689+0000","last_became_peered":"2024-02-10T06:56:47.055689+0000","last_unstale":"2024-02-10T06:57:25.082340+0000","last_undegraded":"2024-02-10T06:57:25.082340+0000","last_fullsized":"2024-02-10T06:57:25.082340+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:12:07.925607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082639+0000","last_change":"2024-02-10T06:56:49.072694+0000","last_active":"2024-02-10T06:57:25.082639+0000","last_peered":"2024-02-10T06:57:25.082639+0000","last_clean":"2024-02-10T06:57:25.082639+0000","last_became_active":"2024-02-10T06:56:49.072366+0000","last_became_peered":"2024-02-10T06:56:49.072366+0000","last_unstale":"2024-02-10T06:57:25.082639+0000","last_undegraded":"2024-02-10T06:57:25.082639+0000","last_fullsized":"2024-02-10T06:57:25.082639+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:50:26.574275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042479+0000","last_change":"2024-02-10T06:56:51.074290+0000","last_active":"2024-02-10T06:57:12.042479+0000","last_peered":"2024-02-10T06:57:12.042479+0000","last_clean":"2024-02-10T06:57:12.042479+0000","last_became_active":"2024-02-10T06:56:51.073900+0000","last_became_peered":"2024-02-10T06:56:51.073900+0000","last_unstale":"2024-02-10T06:57:12.042479+0000","last_undegraded":"2024-02-10T06:57:12.042479+0000","last_fullsized":"2024-02-10T06:57:12.042479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:53:07.511838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044506+0000","last_change":"2024-02-10T06:56:45.034397+0000","last_active":"2024-02-10T06:57:12.044506+0000","last_peered":"2024-02-10T06:57:12.044506+0000","last_clean":"2024-02-10T06:57:12.044506+0000","last_became_active":"2024-02-10T06:56:45.034092+0000","last_became_peered":"2024-02-10T06:56:45.034092+0000","last_unstale":"2024-02-10T06:57:12.044506+0000","last_undegraded":"2024-02-10T06:57:12.044506+0000","last_fullsized":"2024-02-10T06:57:12.044506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:03:10.662061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045481+0000","last_change":"2024-02-10T06:56:47.056469+0000","last_active":"2024-02-10T06:57:12.045481+0000","last_peered":"2024-02-10T06:57:12.045481+0000","last_clean":"2024-02-10T06:57:12.045481+0000","last_became_active":"2024-02-10T06:56:47.055821+0000","last_became_peered":"2024-02-10T06:56:47.055821+0000","last_unstale":"2024-02-10T06:57:12.045481+0000","last_undegraded":"2024-02-10T06:57:12.045481+0000","last_fullsized":"2024-02-10T06:57:12.045481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:25:32.299560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082264+0000","last_change":"2024-02-10T06:56:49.073132+0000","last_active":"2024-02-10T06:57:25.082264+0000","last_peered":"2024-02-10T06:57:25.082264+0000","last_clean":"2024-02-10T06:57:25.082264+0000","last_became_active":"2024-02-10T06:56:49.072821+0000","last_became_peered":"2024-02-10T06:56:49.072821+0000","last_unstale":"2024-02-10T06:57:25.082264+0000","last_undegraded":"2024-02-10T06:57:25.082264+0000","last_fullsized":"2024-02-10T06:57:25.082264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:03:59.935170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046150+0000","last_change":"2024-02-10T06:56:51.073446+0000","last_active":"2024-02-10T06:57:12.046150+0000","last_peered":"2024-02-10T06:57:12.046150+0000","last_clean":"2024-02-10T06:57:12.046150+0000","last_became_active":"2024-02-10T06:56:51.073097+0000","last_became_peered":"2024-02-10T06:56:51.073097+0000","last_unstale":"2024-02-10T06:57:12.046150+0000","last_undegraded":"2024-02-10T06:57:12.046150+0000","last_fullsized":"2024-02-10T06:57:12.046150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:18:35.460329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044482+0000","last_change":"2024-02-10T06:56:45.033653+0000","last_active":"2024-02-10T06:57:12.044482+0000","last_peered":"2024-02-10T06:57:12.044482+0000","last_clean":"2024-02-10T06:57:12.044482+0000","last_became_active":"2024-02-10T06:56:45.033278+0000","last_became_peered":"2024-02-10T06:56:45.033278+0000","last_unstale":"2024-02-10T06:57:12.044482+0000","last_undegraded":"2024-02-10T06:57:12.044482+0000","last_fullsized":"2024-02-10T06:57:12.044482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:14:56.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044280+0000","last_change":"2024-02-10T06:56:47.055445+0000","last_active":"2024-02-10T06:57:12.044280+0000","last_peered":"2024-02-10T06:57:12.044280+0000","last_clean":"2024-02-10T06:57:12.044280+0000","last_became_active":"2024-02-10T06:56:47.055124+0000","last_became_peered":"2024-02-10T06:56:47.055124+0000","last_unstale":"2024-02-10T06:57:12.044280+0000","last_undegraded":"2024-02-10T06:57:12.044280+0000","last_fullsized":"2024-02-10T06:57:12.044280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:42:13.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045878+0000","last_change":"2024-02-10T06:56:49.073535+0000","last_active":"2024-02-10T06:57:12.045878+0000","last_peered":"2024-02-10T06:57:12.045878+0000","last_clean":"2024-02-10T06:57:12.045878+0000","last_became_active":"2024-02-10T06:56:49.073180+0000","last_became_peered":"2024-02-10T06:56:49.073180+0000","last_unstale":"2024-02-10T06:57:12.045878+0000","last_undegraded":"2024-02-10T06:57:12.045878+0000","last_fullsized":"2024-02-10T06:57:12.045878+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:05:48.646235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043343+0000","last_change":"2024-02-10T06:56:51.076745+0000","last_active":"2024-02-10T06:57:12.043343+0000","last_peered":"2024-02-10T06:57:12.043343+0000","last_clean":"2024-02-10T06:57:12.043343+0000","last_became_active":"2024-02-10T06:56:51.076514+0000","last_became_peered":"2024-02-10T06:56:51.076514+0000","last_unstale":"2024-02-10T06:57:12.043343+0000","last_undegraded":"2024-02-10T06:57:12.043343+0000","last_fullsized":"2024-02-10T06:57:12.043343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:11:34.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082561+0000","last_change":"2024-02-10T06:56:49.065784+0000","last_active":"2024-02-10T06:57:25.082561+0000","last_peered":"2024-02-10T06:57:25.082561+0000","last_clean":"2024-02-10T06:57:25.082561+0000","last_became_active":"2024-02-10T06:56:49.065397+0000","last_became_peered":"2024-02-10T06:56:49.065397+0000","last_unstale":"2024-02-10T06:57:25.082561+0000","last_undegraded":"2024-02-10T06:57:25.082561+0000","last_fullsized":"2024-02-10T06:57:25.082561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:13:05.942893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042678+0000","last_change":"2024-02-10T06:56:45.035055+0000","last_active":"2024-02-10T06:57:12.042678+0000","last_peered":"2024-02-10T06:57:12.042678+0000","last_clean":"2024-02-10T06:57:12.042678+0000","last_became_active":"2024-02-10T06:56:45.033337+0000","last_became_peered":"2024-02-10T06:56:45.033337+0000","last_unstale":"2024-02-10T06:57:12.042678+0000","last_undegraded":"2024-02-10T06:57:12.042678+0000","last_fullsized":"2024-02-10T06:57:12.042678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:41:17.064891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046461+0000","last_change":"2024-02-10T06:56:47.048380+0000","last_active":"2024-02-10T06:57:12.046461+0000","last_peered":"2024-02-10T06:57:12.046461+0000","last_clean":"2024-02-10T06:57:12.046461+0000","last_became_active":"2024-02-10T06:56:47.047807+0000","last_became_peered":"2024-02-10T06:56:47.047807+0000","last_unstale":"2024-02-10T06:57:12.046461+0000","last_undegraded":"2024-02-10T06:57:12.046461+0000","last_fullsized":"2024-02-10T06:57:12.046461+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:33:10.863385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047813+0000","last_change":"2024-02-10T06:56:51.074394+0000","last_active":"2024-02-10T06:57:12.047813+0000","last_peered":"2024-02-10T06:57:12.047813+0000","last_clean":"2024-02-10T06:57:12.047813+0000","last_became_active":"2024-02-10T06:56:51.074082+0000","last_became_peered":"2024-02-10T06:56:51.074082+0000","last_unstale":"2024-02-10T06:57:12.047813+0000","last_undegraded":"2024-02-10T06:57:12.047813+0000","last_fullsized":"2024-02-10T06:57:12.047813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:09:38.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045050+0000","last_change":"2024-02-10T06:56:49.066143+0000","last_active":"2024-02-10T06:57:12.045050+0000","last_peered":"2024-02-10T06:57:12.045050+0000","last_clean":"2024-02-10T06:57:12.045050+0000","last_became_active":"2024-02-10T06:56:49.065822+0000","last_became_peered":"2024-02-10T06:56:49.065822+0000","last_unstale":"2024-02-10T06:57:12.045050+0000","last_undegraded":"2024-02-10T06:57:12.045050+0000","last_fullsized":"2024-02-10T06:57:12.045050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:13:41.226479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041960+0000","last_change":"2024-02-10T06:56:45.034599+0000","last_active":"2024-02-10T06:57:12.041960+0000","last_peered":"2024-02-10T06:57:12.041960+0000","last_clean":"2024-02-10T06:57:12.041960+0000","last_became_active":"2024-02-10T06:56:45.032594+0000","last_became_peered":"2024-02-10T06:56:45.032594+0000","last_unstale":"2024-02-10T06:57:12.041960+0000","last_undegraded":"2024-02-10T06:57:12.041960+0000","last_fullsized":"2024-02-10T06:57:12.041960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:58.721446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.045944+0000","last_change":"2024-02-10T06:56:47.052044+0000","last_active":"2024-02-10T06:57:12.045944+0000","last_peered":"2024-02-10T06:57:12.045944+0000","last_clean":"2024-02-10T06:57:12.045944+0000","last_became_active":"2024-02-10T06:56:47.051538+0000","last_became_peered":"2024-02-10T06:56:47.051538+0000","last_unstale":"2024-02-10T06:57:12.045944+0000","last_undegraded":"2024-02-10T06:57:12.045944+0000","last_fullsized":"2024-02-10T06:57:12.045944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:12:09.329537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047377+0000","last_change":"2024-02-10T06:56:51.074053+0000","last_active":"2024-02-10T06:57:12.047377+0000","last_peered":"2024-02-10T06:57:12.047377+0000","last_clean":"2024-02-10T06:57:12.047377+0000","last_became_active":"2024-02-10T06:56:51.073771+0000","last_became_peered":"2024-02-10T06:56:51.073771+0000","last_unstale":"2024-02-10T06:57:12.047377+0000","last_undegraded":"2024-02-10T06:57:12.047377+0000","last_fullsized":"2024-02-10T06:57:12.047377+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:16:32.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047015+0000","last_change":"2024-02-10T06:56:45.034304+0000","last_active":"2024-02-10T06:57:12.047015+0000","last_peered":"2024-02-10T06:57:12.047015+0000","last_clean":"2024-02-10T06:57:12.047015+0000","last_became_active":"2024-02-10T06:56:45.033627+0000","last_became_peered":"2024-02-10T06:56:45.033627+0000","last_unstale":"2024-02-10T06:57:12.047015+0000","last_undegraded":"2024-02-10T06:57:12.047015+0000","last_fullsized":"2024-02-10T06:57:12.047015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:06:46.448014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042122+0000","last_change":"2024-02-10T06:56:47.054188+0000","last_active":"2024-02-10T06:57:12.042122+0000","last_peered":"2024-02-10T06:57:12.042122+0000","last_clean":"2024-02-10T06:57:12.042122+0000","last_became_active":"2024-02-10T06:56:47.053744+0000","last_became_peered":"2024-02-10T06:56:47.053744+0000","last_unstale":"2024-02-10T06:57:12.042122+0000","last_undegraded":"2024-02-10T06:57:12.042122+0000","last_fullsized":"2024-02-10T06:57:12.042122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:38:23.171411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043515+0000","last_change":"2024-02-10T06:56:49.065914+0000","last_active":"2024-02-10T06:57:12.043515+0000","last_peered":"2024-02-10T06:57:12.043515+0000","last_clean":"2024-02-10T06:57:12.043515+0000","last_became_active":"2024-02-10T06:56:49.065524+0000","last_became_peered":"2024-02-10T06:56:49.065524+0000","last_unstale":"2024-02-10T06:57:12.043515+0000","last_undegraded":"2024-02-10T06:57:12.043515+0000","last_fullsized":"2024-02-10T06:57:12.043515+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:29:14.739985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042417+0000","last_change":"2024-02-10T06:56:51.078882+0000","last_active":"2024-02-10T06:57:12.042417+0000","last_peered":"2024-02-10T06:57:12.042417+0000","last_clean":"2024-02-10T06:57:12.042417+0000","last_became_active":"2024-02-10T06:56:51.077581+0000","last_became_peered":"2024-02-10T06:56:51.077581+0000","last_unstale":"2024-02-10T06:57:12.042417+0000","last_undegraded":"2024-02-10T06:57:12.042417+0000","last_fullsized":"2024-02-10T06:57:12.042417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:09:50.115481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:25.082606+0000","last_change":"2024-02-10T06:56:49.066186+0000","last_active":"2024-02-10T06:57:25.082606+0000","last_peered":"2024-02-10T06:57:25.082606+0000","last_clean":"2024-02-10T06:57:25.082606+0000","last_became_active":"2024-02-10T06:56:49.065864+0000","last_became_peered":"2024-02-10T06:56:49.065864+0000","last_unstale":"2024-02-10T06:57:25.082606+0000","last_undegraded":"2024-02-10T06:57:25.082606+0000","last_fullsized":"2024-02-10T06:57:25.082606+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:13:03.832075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047832+0000","last_change":"2024-02-10T06:56:45.034342+0000","last_active":"2024-02-10T06:57:12.047832+0000","last_peered":"2024-02-10T06:57:12.047832+0000","last_clean":"2024-02-10T06:57:12.047832+0000","last_became_active":"2024-02-10T06:56:45.033601+0000","last_became_peered":"2024-02-10T06:56:45.033601+0000","last_unstale":"2024-02-10T06:57:12.047832+0000","last_undegraded":"2024-02-10T06:57:12.047832+0000","last_fullsized":"2024-02-10T06:57:12.047832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:28.090435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043375+0000","last_change":"2024-02-10T06:56:47.049326+0000","last_active":"2024-02-10T06:57:12.043375+0000","last_peered":"2024-02-10T06:57:12.043375+0000","last_clean":"2024-02-10T06:57:12.043375+0000","last_became_active":"2024-02-10T06:56:47.048841+0000","last_became_peered":"2024-02-10T06:56:47.048841+0000","last_unstale":"2024-02-10T06:57:12.043375+0000","last_undegraded":"2024-02-10T06:57:12.043375+0000","last_fullsized":"2024-02-10T06:57:12.043375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:11:03.763961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047088+0000","last_change":"2024-02-10T06:56:51.073441+0000","last_active":"2024-02-10T06:57:12.047088+0000","last_peered":"2024-02-10T06:57:12.047088+0000","last_clean":"2024-02-10T06:57:12.047088+0000","last_became_active":"2024-02-10T06:56:51.073108+0000","last_became_peered":"2024-02-10T06:56:51.073108+0000","last_unstale":"2024-02-10T06:57:12.047088+0000","last_undegraded":"2024-02-10T06:57:12.047088+0000","last_fullsized":"2024-02-10T06:57:12.047088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:10:48.590841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.047665+0000","last_change":"2024-02-10T06:56:45.034694+0000","last_active":"2024-02-10T06:57:12.047665+0000","last_peered":"2024-02-10T06:57:12.047665+0000","last_clean":"2024-02-10T06:57:12.047665+0000","last_became_active":"2024-02-10T06:56:45.034205+0000","last_became_peered":"2024-02-10T06:56:45.034205+0000","last_unstale":"2024-02-10T06:57:12.047665+0000","last_undegraded":"2024-02-10T06:57:12.047665+0000","last_fullsized":"2024-02-10T06:57:12.047665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:48:19.301919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044179+0000","last_change":"2024-02-10T06:56:47.052294+0000","last_active":"2024-02-10T06:57:12.044179+0000","last_peered":"2024-02-10T06:57:12.044179+0000","last_clean":"2024-02-10T06:57:12.044179+0000","last_became_active":"2024-02-10T06:56:47.051956+0000","last_became_peered":"2024-02-10T06:56:47.051956+0000","last_unstale":"2024-02-10T06:57:12.044179+0000","last_undegraded":"2024-02-10T06:57:12.044179+0000","last_fullsized":"2024-02-10T06:57:12.044179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:37:38.445332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042839+0000","last_change":"2024-02-10T06:56:49.065447+0000","last_active":"2024-02-10T06:57:12.042839+0000","last_peered":"2024-02-10T06:57:12.042839+0000","last_clean":"2024-02-10T06:57:12.042839+0000","last_became_active":"2024-02-10T06:56:49.065052+0000","last_became_peered":"2024-02-10T06:56:49.065052+0000","last_unstale":"2024-02-10T06:57:12.042839+0000","last_undegraded":"2024-02-10T06:57:12.042839+0000","last_fullsized":"2024-02-10T06:57:12.042839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:43.610849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041161+0000","last_change":"2024-02-10T06:56:51.077633+0000","last_active":"2024-02-10T06:57:12.041161+0000","last_peered":"2024-02-10T06:57:12.041161+0000","last_clean":"2024-02-10T06:57:12.041161+0000","last_became_active":"2024-02-10T06:56:51.077158+0000","last_became_peered":"2024-02-10T06:56:51.077158+0000","last_unstale":"2024-02-10T06:57:12.041161+0000","last_undegraded":"2024-02-10T06:57:12.041161+0000","last_fullsized":"2024-02-10T06:57:12.041161+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:19:27.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.048006+0000","last_change":"2024-02-10T06:56:45.037343+0000","last_active":"2024-02-10T06:57:12.048006+0000","last_peered":"2024-02-10T06:57:12.048006+0000","last_clean":"2024-02-10T06:57:12.048006+0000","last_became_active":"2024-02-10T06:56:45.037072+0000","last_became_peered":"2024-02-10T06:56:45.037072+0000","last_unstale":"2024-02-10T06:57:12.048006+0000","last_undegraded":"2024-02-10T06:57:12.048006+0000","last_fullsized":"2024-02-10T06:57:12.048006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:04:58.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.043812+0000","last_change":"2024-02-10T06:56:47.050308+0000","last_active":"2024-02-10T06:57:12.043812+0000","last_peered":"2024-02-10T06:57:12.043812+0000","last_clean":"2024-02-10T06:57:12.043812+0000","last_became_active":"2024-02-10T06:56:47.049811+0000","last_became_peered":"2024-02-10T06:56:47.049811+0000","last_unstale":"2024-02-10T06:57:12.043812+0000","last_undegraded":"2024-02-10T06:57:12.043812+0000","last_fullsized":"2024-02-10T06:57:12.043812+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:33:18.649222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.041918+0000","last_change":"2024-02-10T06:56:49.071583+0000","last_active":"2024-02-10T06:57:12.041918+0000","last_peered":"2024-02-10T06:57:12.041918+0000","last_clean":"2024-02-10T06:57:12.041918+0000","last_became_active":"2024-02-10T06:56:49.071002+0000","last_became_peered":"2024-02-10T06:56:49.071002+0000","last_unstale":"2024-02-10T06:57:12.041918+0000","last_undegraded":"2024-02-10T06:57:12.041918+0000","last_fullsized":"2024-02-10T06:57:12.041918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:39:05.925492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.042362+0000","last_change":"2024-02-10T06:56:51.074826+0000","last_active":"2024-02-10T06:57:12.042362+0000","last_peered":"2024-02-10T06:57:12.042362+0000","last_clean":"2024-02-10T06:57:12.042362+0000","last_became_active":"2024-02-10T06:56:51.074474+0000","last_became_peered":"2024-02-10T06:56:51.074474+0000","last_unstale":"2024-02-10T06:57:12.042362+0000","last_undegraded":"2024-02-10T06:57:12.042362+0000","last_fullsized":"2024-02-10T06:57:12.042362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:50.057084+0000","last_clean_scrub_stamp":"2024-02-10T06:56:50.057084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:09:10.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044406+0000","last_change":"2024-02-10T06:56:45.033915+0000","last_active":"2024-02-10T06:57:12.044406+0000","last_peered":"2024-02-10T06:57:12.044406+0000","last_clean":"2024-02-10T06:57:12.044406+0000","last_became_active":"2024-02-10T06:56:45.032773+0000","last_became_peered":"2024-02-10T06:56:45.032773+0000","last_unstale":"2024-02-10T06:57:12.044406+0000","last_undegraded":"2024-02-10T06:57:12.044406+0000","last_fullsized":"2024-02-10T06:57:12.044406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:44.010363+0000","last_clean_scrub_stamp":"2024-02-10T06:56:44.010363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:29:30.600547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.044546+0000","last_change":"2024-02-10T06:56:47.051281+0000","last_active":"2024-02-10T06:57:12.044546+0000","last_peered":"2024-02-10T06:57:12.044546+0000","last_clean":"2024-02-10T06:57:12.044546+0000","last_became_active":"2024-02-10T06:56:47.050712+0000","last_became_peered":"2024-02-10T06:56:47.050712+0000","last_unstale":"2024-02-10T06:57:12.044546+0000","last_undegraded":"2024-02-10T06:57:12.044546+0000","last_fullsized":"2024-02-10T06:57:12.044546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:46.024051+0000","last_clean_scrub_stamp":"2024-02-10T06:56:46.024051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:08:46.266344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T06:57:12.046333+0000","last_change":"2024-02-10T06:56:49.075132+0000","last_active":"2024-02-10T06:57:12.046333+0000","last_peered":"2024-02-10T06:57:12.046333+0000","last_clean":"2024-02-10T06:57:12.046333+0000","last_became_active":"2024-02-10T06:56:49.074852+0000","last_became_peered":"2024-02-10T06:56:49.074852+0000","last_unstale":"2024-02-10T06:57:12.046333+0000","last_undegraded":"2024-02-10T06:57:12.046333+0000","last_fullsized":"2024-02-10T06:57:12.046333+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:56:48.040518+0000","last_clean_scrub_stamp":"2024-02-10T06:56:48.040518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:31:31.579312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411343,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738393,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 06:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.627}]},{"osd":2,"last update":"Sat Feb 10 06:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T06:57:37.662 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-10T06:57:37.662 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-10T06:57:37.662 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-10T06:57:37.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- ceph health --format=json 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:57:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: Deploying daemon grafana.a on smithi136 2024-02-10T06:57:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:57:37.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:37 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:37.901 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:38.284 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:38 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:38.088Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000445468s 2024-02-10T06:57:39.458 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:39 smithi076 ceph-mon[25700]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:39.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:39 smithi136 ceph-mon[29743]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:39 smithi189 ceph-mon[30417]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:39.927 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-10T06:57:39.949 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-10T06:57:40.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:40 smithi076 ceph-mon[25700]: from='client.? 172.21.15.76:0/588320964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:57:40.635 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:40 smithi136 ceph-mon[29743]: from='client.? 172.21.15.76:0/588320964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:57:40.651 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-10T06:57:40.652 INFO:tasks.cephadm:Setup complete, yielding 2024-02-10T06:57:40.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-10T06:57:40.661 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-02-10T06:57:40.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-10T06:57:40.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:40 smithi189 ceph-mon[30417]: from='client.? 172.21.15.76:0/588320964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:57:40.897 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:41.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:41 smithi136 ceph-mon[29743]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:41.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:41 smithi189 ceph-mon[30417]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:42.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:41 smithi076 ceph-mon[25700]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:167 167 2024-02-10T06:57:42.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:42 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:42.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:42 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:42.941 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:42 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:43.255 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch status' 2024-02-10T06:57:43.491 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:43.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:43 smithi136 ceph-mon[29743]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:43.902 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:43 smithi076 ceph-mon[25700]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:43 smithi189 ceph-mon[30417]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:45.366 INFO:teuthology.orchestra.run.smithi076.stdout:Backend: cephadm 2024-02-10T06:57:45.366 INFO:teuthology.orchestra.run.smithi076.stdout:Available: Yes 2024-02-10T06:57:45.366 INFO:teuthology.orchestra.run.smithi076.stdout:Paused: No 2024-02-10T06:57:45.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:45 smithi136 ceph-mon[29743]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:45.905 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:45 smithi076 ceph-mon[25700]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:45.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:45 smithi189 ceph-mon[30417]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:45.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-10T06:57:46.167 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:46.284 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:57:46 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:57:46.089Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00120688s 2024-02-10T06:57:46.747 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:46 smithi136 ceph-mon[29743]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:46.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:46 smithi189 ceph-mon[30417]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:46.926 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:46 smithi076 ceph-mon[25700]: from='client.14523 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:47.284 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:46 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:57:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:47.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:47 smithi136 ceph-mon[29743]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:47.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:47 smithi189 ceph-mon[30417]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:48.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:47 smithi076 ceph-mon[25700]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:48.067 INFO:teuthology.orchestra.run.smithi076.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:alertmanager.a smithi076 *:9093,9094 starting - - - - 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.x smithi136 *:8443,8765 running (2m) 33s ago 2m 447M - 19.0.0-1361-g290212a1 c328ab9dcfc7 7d875dcc2731 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.y smithi076 *:9283,8765 running (4m) 29s ago 4m 476M - 19.0.0-1361-g290212a1 c328ab9dcfc7 02c860c2f4c5 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.z smithi189 *:8443,8765 running (2m) 34s ago 2m 447M - 19.0.0-1361-g290212a1 c328ab9dcfc7 62ba409fd3a8 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mon.a smithi076 running (4m) 29s ago 4m 51.3M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 dae064449349 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mon.b smithi136 running (2m) 33s ago 2m 41.6M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 070be2e33506 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:mon.c smithi189 running (2m) 34s ago 2m 42.9M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 f48c93b90bd3 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0 smithi076 running (2m) 29s ago 2m 57.4M 15.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 7b2d007b0314 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1 smithi136 running (107s) 33s ago 106s 60.2M 15.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 743d82aa9f17 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2 smithi189 running (83s) 34s ago 82s 61.8M 16.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 9deb1c04aafc 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:prometheus.a smithi136 *:9095 running (44s) 33s ago 44s 26.9M - 2.43.0 a07b618ecd1d e22754816e21 2024-02-10T06:57:48.068 INFO:teuthology.orchestra.run.smithi076.stdout:rgw.foo.a smithi076 *:80 running (65s) 29s ago 64s 76.5M - 19.0.0-1361-g290212a1 c328ab9dcfc7 28ab6ea5bc31 2024-02-10T06:57:48.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-10T06:57:48.959 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:49.136 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 podman[42754]: 2024-02-10 06:57:48.956036885 +0000 UTC m=+0.016951101 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-10T06:57:49.424 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 podman[42754]: 2024-02-10 06:57:49.268130939 +0000 UTC m=+0.329045148 container init 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a, release=239.1651231664, com.redhat.component=ubi8-container, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, io.buildah.version=1.32.0, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, distribution-scope=public, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-10T06:57:49.424 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 podman[42754]: 2024-02-10 06:57:49.273769428 +0000 UTC m=+0.334683623 container start 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a, io.openshift.expose-services=, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , description=Ceph Grafana Container, vendor=Red Hat, Inc., io.buildah.version=1.32.0, distribution-scope=public, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, name=ubi8) 2024-02-10T06:57:49.424 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 bash[42754]: 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e 2024-02-10T06:57:49.425 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 systemd[1]: Started Ceph grafana.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.675 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424654322Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424900826Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424932727Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424948199Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424959184Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424968438Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424976959Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424986924Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.424997324Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.425007316Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-10T06:57:49.676 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.425017137Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.425026589Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.425036828Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=settings t=2024-02-10T06:57:49.425046912Z level=info msg="App mode production" 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=sqlstore t=2024-02-10T06:57:49.425103441Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=sqlstore t=2024-02-10T06:57:49.425127941Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.426172473Z level=info msg="Starting DB migrations" 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.426248886Z level=info msg="Executing migration" id="create migration_log table" 2024-02-10T06:57:49.677 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.426739033Z level=info msg="Executing migration" id="create user table" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.427202013Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.427776216Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.428320228Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.428908157Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.429301088Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.429974919Z level=info msg="Executing migration" id="create user table v2" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.43038206Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.430769253Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.431209218Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.431560746Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.431883689Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-10T06:57:49.678 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.432298172Z level=info msg="Executing migration" id="Update user table charset" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.432559784Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.43293969Z level=info msg="Executing migration" id="Add missing user data" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.433278814Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.433660692Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.434123477Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.434604323Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.435964594Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.436402359Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.436847899Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.43941696Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.439867857Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.440302918Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.442885408Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-10T06:57:49.679 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.443354565Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.445444694Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.445882039Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.446307842Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.449175059Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.449734587Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.451681747Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.454317883Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.454735319Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.457309798Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.460003035Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.462556339Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.465056645Z level=info msg="Executing migration" id="create star table" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.46758443Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.470299153Z level=info msg="Executing migration" id="create org table v1" 2024-02-10T06:57:49.680 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.472807144Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.47538867Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.47787884Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.480390972Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.482775145Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.483351821Z level=info msg="Executing migration" id="Update org table charset" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.485892247Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.486227144Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.488557593Z level=info msg="Executing migration" id="create dashboard table" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.489138694Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.490902636Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.493497097Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.496049858Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.498632265Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-10T06:57:49.681 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.500860973Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-10T06:57:49.682 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.503403022Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-10T06:57:49.682 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.505989525Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-10T06:57:49.682 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.508572506Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.511003016Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.513464321Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.516118045Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.518738863Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.519827068Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.520791431Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.52179142Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.522488544Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.523429863Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.52386167Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.524527549Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-10T06:57:49.683 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.525568478Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.526422752Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.527347293Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.528438347Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.52931713Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.530259493Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.531320223Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.532285779Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.533177134Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.53425691Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.535155826Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.536000851Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.536943566Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.537881798Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-10T06:57:49.684 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.538848436Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.539794987Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.540802447Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.541699569Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.542701072Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.54366381Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.544643534Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.545534396Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.546499852Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.54748091Z level=info msg="Executing migration" id="create data_source table" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.548552162Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.549391586Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.550444312Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.55118235Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-10T06:57:49.685 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.552159158Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.553146522Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.554097072Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.555049407Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.555978246Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.556904473Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.557876638Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.56059494Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.563356167Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.566083875Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.568758186Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.571526646Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.57411729Z level=info msg="Executing migration" id="Add uid column" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.576751312Z level=info msg="Executing migration" id="Update uid value" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.579654786Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-10T06:57:49.686 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.582353886Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.585063977Z level=info msg="Executing migration" id="create api_key table" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.587728527Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.590265788Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.592987124Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.595673839Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.598326851Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.600983455Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.603619997Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.604746739Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.607610423Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.610260871Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.61301791Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.613497899Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-10T06:57:49.687 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.6160873Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.616465553Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.618788699Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.619553523Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.621456073Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.621861746Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.624704501Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.627187807Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.629958648Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.630446587Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.632913116Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.633356253Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.635693078Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.638080277Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-10T06:57:49.688 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.638499498Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.6411046Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.64176256Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.643671363Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.646341961Z level=info msg="Executing migration" id="create quota table v1" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.648674207Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.65116455Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.653540321Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.655861127Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.658408943Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.660956927Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.663568666Z level=info msg="Executing migration" id="create session table" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.666053851Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.668277547Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.670529825Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-10T06:57:49.689 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.673022433Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-10T06:57:49.690 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.675468276Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-10T06:57:49.690 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.677670619Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-10T06:57:49.690 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.680154825Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-10T06:57:49.690 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.682527927Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:49 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.684962247Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.68747489Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.689867504Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.692163678Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.694643094Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-10T06:57:49.928 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.697255417Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.699888562Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.702454851Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.70496297Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.707245517Z level=info msg="Executing migration" id="create alert table v1" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.709806446Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.712146234Z level=info msg="Executing migration" id="add index alert state" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.714593059Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.717106339Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.719773017Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.720794466Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.721828454Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.72350114Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-10T06:57:49.929 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.724549405Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-10T06:57:49.930 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.725620756Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-10T06:57:49.930 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.726674876Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-10T06:57:49.930 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.727643356Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.728547293Z level=info msg="Executing migration" id="Add column is_default" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.729708803Z level=info msg="Executing migration" id="Add column frequency" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.73061454Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.731576379Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.732543177Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.733471549Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.734544784Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.735018991Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.735413493Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.735828451Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.736212796Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.737184862Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.738079097Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-10T06:57:49.931 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.738891917Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.739894745Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.740798936Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.741591161Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.742458529Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.743103805Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.743410276Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.744358001Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.744807069Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.745742658Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.746238765Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.747314123Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.750077248Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-10T06:57:49.932 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.752749809Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.755254561Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.757989685Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.760620397Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.76328629Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.766069423Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.768529369Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.771196976Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.773756654Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.776404976Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.779081607Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.781724235Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.784232725Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.787012534Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-10T06:57:49.933 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.789618356Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.792229071Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.794926001Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.797693296Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.800111886Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.802852725Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.805285049Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.807955518Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.810654981Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.813082397Z level=info msg="Executing migration" id="Move region to single row" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.815716612Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.818383337Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.821050429Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.823689772Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-10T06:57:49.934 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.826344289Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.828861299Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.831612949Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.834202647Z level=info msg="Executing migration" id="create test_data table" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.836880628Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.837503194Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.84014988Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.842697303Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.845058615Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.847707878Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.850315246Z level=info msg="Executing migration" id="create team table" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.853033329Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.855422776Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.857911876Z level=info msg="Executing migration" id="create team member table" 2024-02-10T06:57:49.935 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.860369852Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.862873843Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.865293227Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.868004032Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.870453044Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.873040179Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.875426288Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.878051943Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.880461657Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.883108361Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.883584401Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.886032817Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.886622956Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.887028639Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-10T06:57:49.936 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.887473976Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.887833663Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.888134975Z level=info msg="Executing migration" id="create tag table" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.888512596Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.888918199Z level=info msg="Executing migration" id="create login attempt table" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.889354726Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.889762492Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.89016113Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-10T06:57:49.937 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.89198366Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.892372153Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.892783856Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.89310362Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.893454639Z level=info msg="Executing migration" id="create user auth table" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.893838041Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.894295402Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.894580537Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.895356052Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.896078683Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.89679752Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.897598056Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-10T06:57:49.938 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.898011285Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.898724687Z level=info msg="Executing migration" id="create server_lock table" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.899102296Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.899515658Z level=info msg="Executing migration" id="create user auth token table" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.899917557Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.902300966Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.90272574Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.903156214Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.903926292Z level=info msg="Executing migration" id="create cache_data table" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.904323286Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.904728543Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.905149964Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.905555188Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-10T06:57:49.939 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.905808392Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.906102314Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.906519037Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.909081851Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.909499356Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.909758388Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.911909882Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.914510553Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.916967285Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.917414311Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.920020922Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.922449682Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.925006107Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-10T06:57:49.940 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.925499371Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-10T06:57:49.941 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.928094887Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-10T06:57:49.941 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.928663085Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-10T06:57:49.941 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.930529176Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-10T06:57:49.941 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.931089819Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-10T06:57:50.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:50.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:50.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:50.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:50.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:50.035 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:49 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:50.180 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.933200629Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-10T06:57:50.180 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.933635826Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-10T06:57:50.180 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.934649947Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.93563261Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.936590175Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.937003128Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.940408693Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.943714369Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.944843657Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.945307091Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.946273547Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.947235353Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.948201275Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.948638116Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-10T06:57:50.181 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.949697032Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.950562165Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.951469115Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.952299228Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.953223048Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.954320802Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.955272741Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.956086815Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.95711247Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.958210047Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.959107693Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.959949238Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.960905906Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.961952932Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.962941507Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-10T06:57:50.182 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.963557358Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-10T06:57:50.183 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.96456878Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-10T06:57:50.183 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.965577406Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-10T06:57:50.183 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.966426863Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-10T06:57:50.183 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.967500387Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-10T06:57:50.183 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.96835429Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.969288464Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.970341156Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.971285291Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.97211188Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.97317897Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.974029093Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-10T06:57:50.184 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.974977653Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.975804757Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.976689896Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.977750624Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.97874816Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.979144437Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.980186973Z level=info msg="Executing migration" id="create alert_image table" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.981141733Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.981902755Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.982216022Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.983282361Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.984128793Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.984203796Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.985181758Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-10T06:57:50.185 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.985582303Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.986514482Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.986631665Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.986739253Z level=info msg="alerts found to migrate" alerts=0 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.987130309Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.987558316Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.988501286Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.988912703Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.990025468Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.990993103Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.991836119Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.992200572Z level=info msg="Executing migration" id="create data_keys table" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.993111034Z level=info msg="Executing migration" id="create secrets table" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.994151117Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-10T06:57:50.186 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.998587646Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:49 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:49.999555257Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.000497226Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.004734967Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.008908343Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.009349301Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.010298185Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.013048621Z level=info msg="Executing migration" id="create permission table" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.015674119Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.016114975Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.018980117Z level=info msg="Executing migration" id="create role table" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.021607414Z level=info msg="Executing migration" id="add column display_name" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.024242646Z level=info msg="Executing migration" id="add column group_name" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.025211677Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-10T06:57:50.187 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.027934421Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.030776605Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.033575763Z level=info msg="Executing migration" id="create team role table" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.034041955Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.036851007Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.039468026Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.042129414Z level=info msg="Executing migration" id="create user role table" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.044736819Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.047306922Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.050019494Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.052621737Z level=info msg="Executing migration" id="create builtin role table" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.053101617Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.05596382Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.058654834Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-10T06:57:50.188 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.061370317Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.061904865Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.064575141Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.06722666Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.069777014Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.070293021Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.072840791Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.075599621Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.078308408Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.078803242Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.081589924Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.084025395Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.086658871Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.087127606Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.089880265Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-10T06:57:50.189 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.092289955Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.09488416Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.097522032Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.09997245Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.102597935Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.104993845Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.10747384Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.109902538Z level=info msg="Executing migration" id="add correlation config column" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.112280375Z level=info msg="Executing migration" id="create entity_events table" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.114674244Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.117318733Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.117537985Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.119746743Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:57:50.190 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.119824575Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.122224176Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.124903747Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.12711265Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.129691117Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.132046366Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.134780324Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.137130322Z level=info msg="Executing migration" id="Drop public config table" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.139469369Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.142097916Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.144603229Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.147023397Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.14971617Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.15279468Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-10T06:57:50.191 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.153801141Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.156506668Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.158817724Z level=info msg="Executing migration" id="add share column" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.161201858Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.161633218Z level=info msg="Executing migration" id="create file table" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.16434976Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.167174631Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.167634821Z level=info msg="Executing migration" id="create file_meta table" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.169717958Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.172236868Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.174699518Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.177292282Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.177661915Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.180349948Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-10T06:57:50.192 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.182785521Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.185434026Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.185898731Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.188411597Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.191127342Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.193601445Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.194682512Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-10T06:57:50.636 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.197260409Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.199977532Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.20972998Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.210216237Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.212981393Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.216500634Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.219146008Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.219671462Z level=info msg="Executing migration" id="create folder table" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.222336941Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.224771533Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.227367674Z level=info msg="Executing migration" id="Update folder title length" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.227684244Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=migrator t=2024-02-10T06:57:50.230320467Z level=info msg="migrations completed" performed=485 skipped=0 duration=804.09591ms 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=sqlstore t=2024-02-10T06:57:50.230749388Z level=info msg="Created default organization" 2024-02-10T06:57:50.637 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=plugin.loader t=2024-02-10T06:57:50.287121717Z level=info msg="Plugin registered" pluginID=input 2024-02-10T06:57:50.638 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=plugin.loader t=2024-02-10T06:57:50.315289292Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-10T06:57:50.638 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=plugin.loader t=2024-02-10T06:57:50.315318449Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-10T06:57:50.638 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=secrets t=2024-02-10T06:57:50.315351022Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-10T06:57:50.638 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=query_data t=2024-02-10T06:57:50.3174986Z level=info msg="Query Service initialization" 2024-02-10T06:57:50.638 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=live.push_http t=2024-02-10T06:57:50.319864608Z level=info msg="Live Push Gateway initialization" 2024-02-10T06:57:50.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:50 smithi189 ceph-mon[30417]: Deploying daemon node-exporter.a on smithi076 2024-02-10T06:57:50.989 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:50 smithi076 ceph-mon[25700]: Deploying daemon node-exporter.a on smithi076 2024-02-10T06:57:50.989 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:50 smithi076 systemd[1]: Starting Ceph node-exporter.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:57:51.025 INFO:teuthology.orchestra.run.smithi076.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-10T06:57:51.025 INFO:teuthology.orchestra.run.smithi076.stdout:alertmanager ?:9093,9094 0/1 - 62s smithi076=a;count:1 2024-02-10T06:57:51.025 INFO:teuthology.orchestra.run.smithi076.stdout:grafana ?:3000 0/1 - 60s smithi136=a;count:1 2024-02-10T06:57:51.025 INFO:teuthology.orchestra.run.smithi076.stdout:mgr 3/3 37s ago 2m smithi076=y;smithi136=x;smithi189=z;count:3 2024-02-10T06:57:51.025 INFO:teuthology.orchestra.run.smithi076.stdout:mon 3/3 37s ago 3m smithi076:172.21.15.76=a;smithi136:172.21.15.136=b;smithi189:172.21.15.189=c;count:3 2024-02-10T06:57:51.026 INFO:teuthology.orchestra.run.smithi076.stdout:node-exporter ?:9100 0/3 - 65s smithi076=a;smithi136=b;smithi189=c;count:3 2024-02-10T06:57:51.026 INFO:teuthology.orchestra.run.smithi076.stdout:osd 3 37s ago - 2024-02-10T06:57:51.026 INFO:teuthology.orchestra.run.smithi076.stdout:prometheus ?:9095 1/1 36s ago 68s smithi136=a;count:1 2024-02-10T06:57:51.026 INFO:teuthology.orchestra.run.smithi076.stdout:rgw.foo.a ?:80 1/1 32s ago 67s smithi076=foo.a;count:1 2024-02-10T06:57:51.026 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:50 smithi136 ceph-mon[29743]: Deploying daemon node-exporter.a on smithi076 2024-02-10T06:57:51.284 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:50 smithi076 bash[59495]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=infra.usagestats.collector t=2024-02-10T06:57:51.026969747Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=provisioning.datasources t=2024-02-10T06:57:51.02822021Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=provisioning.datasources t=2024-02-10T06:57:51.049474836Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=provisioning.alerting t=2024-02-10T06:57:51.070681083Z level=info msg="starting to provision alerting" 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=provisioning.alerting t=2024-02-10T06:57:51.070714971Z level=info msg="finished to provision alerting" 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=ngalert.state.manager t=2024-02-10T06:57:51.070797856Z level=info msg="Warming state cache for startup" 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=ngalert.state.manager t=2024-02-10T06:57:51.070943054Z level=info msg="State cache has been initialized" states=0 duration=144.129µs 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=ticker t=2024-02-10T06:57:51.071053507Z level=info msg=starting first_tick=2024-02-10T06:58:00Z 2024-02-10T06:57:51.386 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=ngalert.multiorg.alertmanager t=2024-02-10T06:57:51.071077633Z level=info msg="starting MultiOrg Alertmanager" 2024-02-10T06:57:51.387 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=grafanaStorageLogger t=2024-02-10T06:57:51.071295228Z level=info msg="storage starting" 2024-02-10T06:57:51.387 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a[42769]: logger=http.server t=2024-02-10T06:57:51.073133529Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-10T06:57:51.648 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:51 smithi076 bash[59495]: Getting image source signatures 2024-02-10T06:57:51.648 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:51 smithi076 bash[59495]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T06:57:51.649 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:51 smithi076 bash[59495]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T06:57:51.649 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:51 smithi076 bash[59495]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T06:57:51.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-10T06:57:51.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:51 smithi189 ceph-mon[30417]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:52.033 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:52.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:51 smithi076 ceph-mon[25700]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:52.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:51 smithi136 ceph-mon[29743]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:52.674 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:52 smithi076 ceph-mon[25700]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:52.674 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:52 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:52.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:52 smithi189 ceph-mon[30417]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:52.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:52 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:53.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:52 smithi136 ceph-mon[29743]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:53.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:52 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:53.654 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:53 smithi076 bash[59495]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T06:57:53.654 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:53 smithi076 bash[59495]: Writing manifest to image destination 2024-02-10T06:57:53.909 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:53 smithi076 ceph-mon[25700]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:53.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:53 smithi189 ceph-mon[30417]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:54.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:53 smithi136 ceph-mon[29743]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:54.216 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:53 smithi076 podman[59495]: 2024-02-10 06:57:53.888242634 +0000 UTC m=+3.007441041 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T06:57:54.216 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 podman[59495]: 2024-02-10 06:57:54.103436097 +0000 UTC m=+3.222634538 container create 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 podman[59495]: 2024-02-10 06:57:54.56683522 +0000 UTC m=+3.686033619 container init 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 podman[59495]: 2024-02-10 06:57:54.572889466 +0000 UTC m=+3.692087866 container start 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.576Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.576Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.577Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.577Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.577Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.577Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T06:57:54.902 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T06:57:54.903 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T06:57:54.904 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T06:57:54.905 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a[59699]: ts=2024-02-10T06:57:54.578Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T06:57:54.906 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 bash[59495]: 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 2024-02-10T06:57:55.031 INFO:teuthology.orchestra.run.smithi076.stdout:HOST ADDR LABELS STATUS 2024-02-10T06:57:55.031 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 172.21.15.76 2024-02-10T06:57:55.031 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 172.21.15.136 2024-02-10T06:57:55.031 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 172.21.15.189 2024-02-10T06:57:55.031 INFO:teuthology.orchestra.run.smithi076.stdout:3 hosts in cluster 2024-02-10T06:57:55.284 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:57:54 smithi076 systemd[1]: Started Ceph node-exporter.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:57:55.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-10T06:57:55.883 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:56.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:55 smithi076 ceph-mon[25700]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:56.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:55 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:55 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:55 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:55 smithi076 ceph-mon[25700]: Deploying daemon node-exporter.b on smithi136 2024-02-10T06:57:56.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:55 smithi136 ceph-mon[29743]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:56.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:55 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:55 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:55 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:55 smithi136 ceph-mon[29743]: Deploying daemon node-exporter.b on smithi136 2024-02-10T06:57:56.386 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:56 smithi136 bash[43053]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T06:57:56.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:55 smithi189 ceph-mon[30417]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:56.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:55 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:55 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:55 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:57:56.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:55 smithi189 ceph-mon[30417]: Deploying daemon node-exporter.b on smithi136 2024-02-10T06:57:56.885 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:56 smithi136 bash[43053]: Getting image source signatures 2024-02-10T06:57:56.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:56 smithi136 bash[43053]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T06:57:56.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:56 smithi136 bash[43053]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T06:57:56.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:56 smithi136 bash[43053]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T06:57:57.165 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:57:56 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:57:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:57.166 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:56 smithi076 ceph-mon[25700]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:57.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:56 smithi136 ceph-mon[29743]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:57.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:56 smithi189 ceph-mon[30417]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme1n1 ssd Linux_0bbd64bcd61d913e7401 89.4G Yes 2m ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme2n1 ssd Linux_f0744870f4d973221029 89.4G Yes 2m ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme3n1 ssd Linux_39ac285e0f8e70e9b2c6 89.4G Yes 2m ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme4n1 ssd Linux_8c456841778157842c71 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 109s ago Has a FileSystem, LVM detected 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 /dev/nvme1n1 ssd Linux_fb47b372b904fcaad8a7 89.4G Yes 109s ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 /dev/nvme2n1 ssd Linux_a57f22516f2564befdc8 89.4G Yes 109s ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 /dev/nvme3n1 ssd Linux_28930fd89c0cba5f79c3 89.4G Yes 109s ago 2024-02-10T06:57:57.977 INFO:teuthology.orchestra.run.smithi076.stdout:smithi136 /dev/nvme4n1 ssd Linux_eeb92fdfa6799b14ad7b 89.4G No 109s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T06:57:57.978 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 84s ago Has a FileSystem, LVM detected 2024-02-10T06:57:57.978 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 /dev/nvme1n1 ssd Linux_d4e4981338ece477f21d 89.4G Yes 84s ago 2024-02-10T06:57:57.978 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 /dev/nvme2n1 ssd Linux_9068d91efb19b7abd9cd 89.4G Yes 84s ago 2024-02-10T06:57:57.978 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 /dev/nvme3n1 ssd Linux_e78950c98353815517c4 89.4G Yes 84s ago 2024-02-10T06:57:57.978 INFO:teuthology.orchestra.run.smithi076.stdout:smithi189 /dev/nvme4n1 ssd Linux_83a9f29c9023692046fc 89.4G No 84s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T06:57:58.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:57 smithi076 ceph-mon[25700]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:58.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:57 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:58.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:57 smithi136 ceph-mon[29743]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:58.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:57 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:58.386 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 bash[43053]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T06:57:58.386 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 bash[43053]: Writing manifest to image destination 2024-02-10T06:57:58.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:57 smithi189 ceph-mon[30417]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:57:58.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:57 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:58.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-10T06:57:58.860 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 podman[43053]: 2024-02-10 06:57:58.46875141 +0000 UTC m=+2.428093288 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T06:57:58.860 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 podman[43053]: 2024-02-10 06:57:58.675766338 +0000 UTC m=+2.635108261 container create a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:57:58.860 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 podman[43053]: 2024-02-10 06:57:58.86069976 +0000 UTC m=+2.820041624 container init a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:57:59.070 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:57:59.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-mon[29743]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 podman[43053]: 2024-02-10 06:57:58.866121055 +0000 UTC m=+2.825462925 container start a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.870Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.870Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.870Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.870Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.870Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T06:57:59.136 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T06:57:59.137 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T06:57:59.138 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.871Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T06:57:59.139 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b[43103]: ts=2024-02-10T06:57:58.872Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 bash[43053]: a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea 2024-02-10T06:57:59.140 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:57:58 smithi136 systemd[1]: Started Ceph node-exporter.b for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:57:59.192 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:57:58 smithi076 ceph-mon[25700]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:57:59.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:57:58 smithi189 ceph-mon[30417]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:58:00.225 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:00 smithi076 ceph-mon[25700]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:00.225 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:00 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.225 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:00 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.225 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:00 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:00 smithi136 ceph-mon[29743]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:00.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:00 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:00 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:00 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:00 smithi189 ceph-mon[30417]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:00.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:00 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:00 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:00 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:00.421 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:00 smithi189 bash[44271]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T06:58:00.921 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:00 smithi189 bash[44271]: Getting image source signatures 2024-02-10T06:58:00.921 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:00 smithi189 bash[44271]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T06:58:00.921 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:00 smithi189 bash[44271]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T06:58:00.921 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:00 smithi189 bash[44271]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T06:58:00.957 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: alertmanager 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: alertmanager 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: count: 1 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi076=a 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:56:48.205456Z' 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: ports: 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: - 9093 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: - 9094 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: running: 0 2024-02-10T06:58:00.958 INFO:teuthology.orchestra.run.smithi076.stdout: size: 1 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:events: 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:- 2024-02-10T06:57:36.269662Z service:alertmanager [INFO] "service was created" 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: grafana 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: grafana 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: count: 1 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi136=a 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:spec: 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: anonymous_access: true 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: protocol: https 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:56:50.873938Z' 2024-02-10T06:58:00.959 INFO:teuthology.orchestra.run.smithi076.stdout: ports: 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: - 3000 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: running: 0 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: size: 1 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:events: 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:- 2024-02-10T06:57:49.456873Z service:grafana [INFO] "service was created" 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: mgr 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: mgr 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: count: 3 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi076=y 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi136=x 2024-02-10T06:58:00.960 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi189=z 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:55:12.687410Z' 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: last_refresh: '2024-02-10T06:57:13.747953Z' 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: running: 3 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: size: 3 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: mon 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: mon 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: count: 3 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi076:172.21.15.76=a 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi136:172.21.15.136=b 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi189:172.21.15.189=c 2024-02-10T06:58:00.961 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:54:46.114818Z' 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: last_refresh: '2024-02-10T06:57:13.747830Z' 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: running: 3 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: size: 3 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: node-exporter 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: node-exporter 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: count: 3 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi076=a 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi136=b 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi189=c 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.962 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:56:45.348171Z' 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: ports: 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: - 9100 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: running: 0 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: size: 3 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:events: 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:- 2024-02-10T06:57:59.063092Z service:node-exporter [INFO] "service was created" 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: osd 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: osd 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:unmanaged: true 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:spec: 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: filter_logic: AND 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout: objectstore: bluestore 2024-02-10T06:58:00.963 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: container_image_id: c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: last_refresh: '2024-02-10T06:57:13.748070Z' 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: running: 3 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: size: 3 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: prometheus 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: prometheus 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: count: 1 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi136=a 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:56:42.530139Z' 2024-02-10T06:58:00.964 INFO:teuthology.orchestra.run.smithi076.stdout: last_refresh: '2024-02-10T06:57:14.399793Z' 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: ports: 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: - 9095 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: running: 1 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: size: 1 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:--- 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:service_type: rgw 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:service_id: foo.a 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:service_name: rgw.foo.a 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:placement: 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: count: 1 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: hosts: 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: - smithi076=foo.a 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout:status: 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: created: '2024-02-10T06:56:43.449120Z' 2024-02-10T06:58:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: last_refresh: '2024-02-10T06:57:18.308822Z' 2024-02-10T06:58:00.966 INFO:teuthology.orchestra.run.smithi076.stdout: ports: 2024-02-10T06:58:00.966 INFO:teuthology.orchestra.run.smithi076.stdout: - 80 2024-02-10T06:58:00.966 INFO:teuthology.orchestra.run.smithi076.stdout: running: 1 2024-02-10T06:58:00.966 INFO:teuthology.orchestra.run.smithi076.stdout: size: 1 2024-02-10T06:58:01.284 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:01 smithi076 ceph-mon[25700]: Deploying daemon node-exporter.c on smithi189 2024-02-10T06:58:01.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:01 smithi136 ceph-mon[29743]: Deploying daemon node-exporter.c on smithi189 2024-02-10T06:58:01.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:01 smithi189 ceph-mon[30417]: Deploying daemon node-exporter.c on smithi189 2024-02-10T06:58:01.653 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8085918-c7e0-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-10T06:58:01.894 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:58:02.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-mon[30417]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:02.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-mon[30417]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T06:58:02.171 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:01 smithi189 bash[44271]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T06:58:02.171 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 bash[44271]: Writing manifest to image destination 2024-02-10T06:58:02.269 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:02 smithi076 ceph-mon[25700]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:02.269 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:02 smithi076 ceph-mon[25700]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T06:58:02.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:02 smithi136 ceph-mon[29743]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:02.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:02 smithi136 ceph-mon[29743]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T06:58:02.594 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 podman[44271]: 2024-02-10 06:58:02.235662497 +0000 UTC m=+2.123558857 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T06:58:02.594 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 podman[44271]: 2024-02-10 06:58:02.317494795 +0000 UTC m=+2.205391206 container create 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:58:02.594 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 podman[44271]: 2024-02-10 06:58:02.594336867 +0000 UTC m=+2.482233225 container init 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:58:02.921 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 podman[44271]: 2024-02-10 06:58:02.59971327 +0000 UTC m=+2.487609617 container start 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.603Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.603Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T06:58:02.922 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T06:58:02.923 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T06:58:02.924 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.604Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T06:58:02.925 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c[44322]: ts=2024-02-10T06:58:02.605Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 bash[44271]: 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 2024-02-10T06:58:02.926 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:58:02 smithi189 systemd[1]: Started Ceph node-exporter.c for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:03.843 INFO:teuthology.orchestra.run.smithi076.stdout:osd 3 50s ago - 2024-02-10T06:58:03.997 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:03.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:03 smithi189 ceph-mon[30417]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.034 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.035 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:03 smithi076 ceph-mon[25700]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' 2024-02-10T06:58:04.136 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:03 smithi136 ceph-mon[29743]: from='mgr.14418 172.21.15.76:0/2847483630' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:04.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:04 smithi076 ceph-mon[25700]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:58:04.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:04 smithi189 ceph-mon[30417]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:58:05.135 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:04 smithi136 ceph-mon[29743]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:58:05.289 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-10T06:58:05.299 INFO:tasks.cephadm:Teardown begin 2024-02-10T06:58:05.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:58:05.331 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:58:05.368 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:58:05.399 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-10T06:58:05.399 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T06:58:05.414 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T06:58:05.431 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T06:58:05.454 INFO:tasks.cephadm:Stopping all daemons... 2024-02-10T06:58:05.454 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-10T06:58:05.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a 2024-02-10T06:58:05.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:05 smithi076 systemd[1]: Stopping Ceph mon.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:06.124 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a[25676]: 2024-02-10T06:58:05.902+0000 7f7a6e14c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a[25676]: 2024-02-10T06:58:05.902+0000 7f7a6e14c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:05] ENGINE Bus STOPPING 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:05] ENGINE Bus STOPPED 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:05 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:05] ENGINE Bus STARTING 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:06] ENGINE Serving on http://:::9283 2024-02-10T06:58:06.125 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: [10/Feb/2024:06:58:06] ENGINE Bus STARTED 2024-02-10T06:58:06.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:06 smithi076 podman[60859]: 2024-02-10 06:58:06.124758854 +0000 UTC m=+0.355002769 container died dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-290212a) 2024-02-10T06:58:06.927 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:06 smithi076 podman[60859]: 2024-02-10 06:58:06.794943345 +0000 UTC m=+1.025187259 container cleanup dae0644493493bd1ecd9214b5a8c8638fafa7fef2cae9eadb8be19e5dea56f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:58:06.927 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:06 smithi076 bash[60859]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a 2024-02-10T06:58:07.197 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:06 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: ::ffff:172.21.15.136 - - [10/Feb/2024:06:58:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:07.534 INFO:journalctl@ceph.mon.a.smithi076.stdout:Feb 10 06:58:07 smithi076 bash[61019]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-a 2024-02-10T06:58:07.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.a.service' 2024-02-10T06:58:08.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:08.005 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-10T06:58:08.005 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-10T06:58:08.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.b 2024-02-10T06:58:08.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:08 smithi136 systemd[1]: Stopping Ceph mon.b for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:09.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b[29720]: 2024-02-10T06:58:09.047+0000 7f0be2ba6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:09.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b[29720]: 2024-02-10T06:58:09.047+0000 7f0be2ba6700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-10T06:58:09.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 podman[43497]: 2024-02-10 06:58:09.254237942 +0000 UTC m=+0.311639833 container died 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-290212a) 2024-02-10T06:58:09.885 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 podman[43497]: 2024-02-10 06:58:09.746508124 +0000 UTC m=+0.803910016 container cleanup 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b, RELEASE=main-290212a, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-10T06:58:09.886 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 bash[43497]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b 2024-02-10T06:58:10.385 INFO:journalctl@ceph.mon.b.smithi136.stdout:Feb 10 06:58:09 smithi136 podman[43536]: 2024-02-10 06:58:09.931459761 +0000 UTC m=+0.784582041 container remove 070be2e335066b198d47aa61d97845345779472bf032b645a6058713d9bc0c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-b, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-10T06:58:10.778 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.b.service' 2024-02-10T06:58:10.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:10.848 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-10T06:58:10.848 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-10T06:58:10.848 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.c 2024-02-10T06:58:11.157 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:10 smithi189 systemd[1]: Stopping Ceph mon.c for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-c[30394]: 2024-02-10T06:58:11.156+0000 7ff8ed26f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:11 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-c[30394]: 2024-02-10T06:58:11.156+0000 7ff8ed26f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-10T06:58:11.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:11 smithi189 podman[44771]: 2024-02-10 06:58:11.177731404 +0000 UTC m=+0.139643254 container died f48c93b90bd314736da5d2702c2fff4df90aedb47c504e67d11a551a0fea20be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, CEPH_POINT_RELEASE=) 2024-02-10T06:58:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:11 smithi189 podman[44771]: 2024-02-10 06:58:11.601567671 +0000 UTC m=+0.563479527 container cleanup f48c93b90bd314736da5d2702c2fff4df90aedb47c504e67d11a551a0fea20be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-290212a, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0) 2024-02-10T06:58:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 10 06:58:11 smithi189 bash[44771]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mon-c 2024-02-10T06:58:12.011 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mon.c.service' 2024-02-10T06:58:12.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:12.085 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-10T06:58:12.086 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-10T06:58:12.086 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y 2024-02-10T06:58:12.441 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 systemd[1]: Stopping Ceph mgr.y for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:12.746 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:58:12.438+0000 7f1661e44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:12.746 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y[25956]: 2024-02-10T06:58:12.438+0000 7f1661e44700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-10T06:58:12.746 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 podman[61117]: 2024-02-10 06:58:12.47176583 +0000 UTC m=+0.191343469 container died 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a) 2024-02-10T06:58:13.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 podman[61117]: 2024-02-10 06:58:12.921120568 +0000 UTC m=+0.640698207 container cleanup 02c860c2f4c5dd448c3595be2b76c62820651fbadba9e8ddf79592d6d73c0f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2) 2024-02-10T06:58:13.034 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Feb 10 06:58:12 smithi076 bash[61117]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-y 2024-02-10T06:58:13.482 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.y.service' 2024-02-10T06:58:13.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:13.519 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-10T06:58:13.519 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-10T06:58:13.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x 2024-02-10T06:58:13.865 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:13 smithi136 systemd[1]: Stopping Ceph mgr.x for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:14.136 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:13 smithi136 podman[43853]: 2024-02-10 06:58:13.865128237 +0000 UTC m=+0.141163454 container died 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-10T06:58:14.428 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 podman[43853]: 2024-02-10 06:58:14.249462207 +0000 UTC m=+0.525497424 container cleanup 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-02-10T06:58:14.428 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 bash[43853]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x 2024-02-10T06:58:14.428 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 podman[43872]: 2024-02-10 06:58:14.425697651 +0000 UTC m=+0.551632670 container remove 7d875dcc273150e031f36216f34de35ee400528912b4c378be2afd9f41836c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-10T06:58:14.428 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-02-10T06:58:14.658 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x.service' 2024-02-10T06:58:14.682 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-02-10T06:58:14.683 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 systemd[1]: Stopped Ceph mgr.x for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:14.683 INFO:journalctl@ceph.mgr.x.smithi136.stdout:Feb 10 06:58:14 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.x.service: Consumed 16.491s CPU time. 2024-02-10T06:58:14.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:14.696 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-10T06:58:14.697 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-10T06:58:14.697 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.z 2024-02-10T06:58:15.031 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:58:14 smithi189 systemd[1]: Stopping Ceph mgr.z for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:15.325 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:58:15 smithi189 podman[44902]: 2024-02-10 06:58:15.031255389 +0000 UTC m=+0.142779685 container died 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-10T06:58:15.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:58:15 smithi189 podman[44902]: 2024-02-10 06:58:15.432981322 +0000 UTC m=+0.544505617 container cleanup 62ba409fd3a852f57a5e0080f2c53f9ac5d126b993b6b1c2748a13da471f3a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-10T06:58:15.671 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 10 06:58:15 smithi189 bash[44902]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-mgr-z 2024-02-10T06:58:15.854 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@mgr.z.service' 2024-02-10T06:58:15.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:15.898 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-10T06:58:15.899 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-10T06:58:15.899 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.0 2024-02-10T06:58:16.196 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:15 smithi076 systemd[1]: Stopping Ceph osd.0 for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:16.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:16 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0[44758]: 2024-02-10T06:58:16.195+0000 7f69d1c5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:16.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:16 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0[44758]: 2024-02-10T06:58:16.195+0000 7f69d1c5d700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-10T06:58:16.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:16 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0[44758]: 2024-02-10T06:58:16.195+0000 7f69d1c5d700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T06:58:21.901 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:21 smithi076 podman[61248]: 2024-02-10 06:58:21.621329037 +0000 UTC m=+5.528872732 container died 7b2d007b0314a221484fa175450bd00e302d7106ef9a4e232101599de93d7731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:58:22.284 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61248]: 2024-02-10 06:58:22.08460752 +0000 UTC m=+5.992151215 container cleanup 7b2d007b0314a221484fa175450bd00e302d7106ef9a4e232101599de93d7731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:58:22.284 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 bash[61248]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0 2024-02-10T06:58:22.712 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61270]: 2024-02-10 06:58:22.369313893 +0000 UTC m=+0.738430385 container remove 7b2d007b0314a221484fa175450bd00e302d7106ef9a4e232101599de93d7731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:58:23.034 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61370]: 2024-02-10 06:58:22.712235281 +0000 UTC m=+0.112702238 container create 6c678a61963acbae375dc1e0d4d1aec45ea1592093eeeafbb3f4c443b15c0a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, RELEASE=main-290212a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-10T06:58:23.034 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61370]: 2024-02-10 06:58:22.616830741 +0000 UTC m=+0.017297671 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:58:23.035 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61370]: 2024-02-10 06:58:22.948670335 +0000 UTC m=+0.349137259 container init 6c678a61963acbae375dc1e0d4d1aec45ea1592093eeeafbb3f4c443b15c0a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:58:23.035 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:22 smithi076 podman[61370]: 2024-02-10 06:58:22.954369529 +0000 UTC m=+0.354836445 container start 6c678a61963acbae375dc1e0d4d1aec45ea1592093eeeafbb3f4c443b15c0a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-290212a, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:58:23.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:23 smithi076 podman[61370]: 2024-02-10 06:58:23.039917403 +0000 UTC m=+0.440384330 container attach 6c678a61963acbae375dc1e0d4d1aec45ea1592093eeeafbb3f4c443b15c0a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-290212a) 2024-02-10T06:58:23.534 INFO:journalctl@ceph.osd.0.smithi076.stdout:Feb 10 06:58:23 smithi076 podman[61370]: 2024-02-10 06:58:23.247028533 +0000 UTC m=+0.647495456 container died 6c678a61963acbae375dc1e0d4d1aec45ea1592093eeeafbb3f4c443b15c0a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS) 2024-02-10T06:58:23.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.0.service' 2024-02-10T06:58:23.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:23.869 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-10T06:58:23.870 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-10T06:58:23.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.1 2024-02-10T06:58:24.181 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:23 smithi136 systemd[1]: Stopping Ceph osd.1 for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:24.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:24 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1[34289]: 2024-02-10T06:58:24.179+0000 7f9e64f96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:24.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:24 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1[34289]: 2024-02-10T06:58:24.179+0000 7f9e64f96700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-10T06:58:24.636 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:24 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1[34289]: 2024-02-10T06:58:24.179+0000 7f9e64f96700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T06:58:29.960 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:29 smithi136 podman[43998]: 2024-02-10 06:58:29.674005267 +0000 UTC m=+5.611904374 container died 743d82aa9f173c6c810144e0f32588a51e734b9a5d56b8c7c1721267a9c38689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-10T06:58:30.284 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[43998]: 2024-02-10 06:58:30.059030829 +0000 UTC m=+5.996929919 container cleanup 743d82aa9f173c6c810144e0f32588a51e734b9a5d56b8c7c1721267a9c38689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-10T06:58:30.285 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 bash[43998]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1 2024-02-10T06:58:30.585 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44019]: 2024-02-10 06:58:30.284871959 +0000 UTC m=+0.607155339 container remove 743d82aa9f173c6c810144e0f32588a51e734b9a5d56b8c7c1721267a9c38689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:58:30.863 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44116]: 2024-02-10 06:58:30.585859712 +0000 UTC m=+0.101751172 container create 3ffe26974066d5322199565e46dcd260842c0e7515503825eb28ab241babb379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-10T06:58:30.863 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44116]: 2024-02-10 06:58:30.50211051 +0000 UTC m=+0.018001993 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:58:30.863 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44116]: 2024-02-10 06:58:30.796769997 +0000 UTC m=+0.312661469 container init 3ffe26974066d5322199565e46dcd260842c0e7515503825eb28ab241babb379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:58:30.863 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44116]: 2024-02-10 06:58:30.802026748 +0000 UTC m=+0.317918217 container start 3ffe26974066d5322199565e46dcd260842c0e7515503825eb28ab241babb379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-10T06:58:30.863 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:30 smithi136 podman[44116]: 2024-02-10 06:58:30.863038695 +0000 UTC m=+0.378930175 container attach 3ffe26974066d5322199565e46dcd260842c0e7515503825eb28ab241babb379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-10T06:58:31.135 INFO:journalctl@ceph.osd.1.smithi136.stdout:Feb 10 06:58:31 smithi136 podman[44116]: 2024-02-10 06:58:31.093343573 +0000 UTC m=+0.609235035 container died 3ffe26974066d5322199565e46dcd260842c0e7515503825eb28ab241babb379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a) 2024-02-10T06:58:31.651 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.1.service' 2024-02-10T06:58:31.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:31.693 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-10T06:58:31.693 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-10T06:58:31.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.2 2024-02-10T06:58:31.980 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:31 smithi189 systemd[1]: Stopping Ceph osd.2 for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:32.421 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:31 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2[35654]: 2024-02-10T06:58:31.978+0000 7f94737ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:32.421 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:31 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2[35654]: 2024-02-10T06:58:31.978+0000 7f94737ce700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-10T06:58:32.421 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:31 smithi189 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2[35654]: 2024-02-10T06:58:31.978+0000 7f94737ce700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T06:58:37.659 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:37 smithi189 podman[45049]: 2024-02-10 06:58:37.367818771 +0000 UTC m=+5.481615171 container died 9deb1c04aafc216b2dfef072731e4f6f2e8ff3f4c1eaf6173b67a60da09c8211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-10T06:58:37.659 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:37 smithi189 podman[45049]: 2024-02-10 06:58:37.659127612 +0000 UTC m=+5.772924017 container cleanup 9deb1c04aafc216b2dfef072731e4f6f2e8ff3f4c1eaf6173b67a60da09c8211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:58:37.921 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:37 smithi189 bash[45049]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2 2024-02-10T06:58:37.921 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:37 smithi189 podman[45071]: 2024-02-10 06:58:37.901620072 +0000 UTC m=+0.529982854 container remove 9deb1c04aafc216b2dfef072731e4f6f2e8ff3f4c1eaf6173b67a60da09c8211 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-10T06:58:38.513 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.119078732 +0000 UTC m=+0.018036108 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:58:38.513 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.218896236 +0000 UTC m=+0.117853611 container create 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_BRANCH=HEAD) 2024-02-10T06:58:38.513 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.421629979 +0000 UTC m=+0.320587356 container init 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-10T06:58:38.513 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.427499599 +0000 UTC m=+0.326456979 container start 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=main-290212a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-10T06:58:38.513 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.512482474 +0000 UTC m=+0.411439853 container attach 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:58:38.921 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:38 smithi189 podman[45165]: 2024-02-10 06:58:38.724889327 +0000 UTC m=+0.623846703 container died 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux ) 2024-02-10T06:58:39.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.2.service' 2024-02-10T06:58:39.233 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:39 smithi189 podman[45212]: 2024-02-10 06:58:39.190033223 +0000 UTC m=+0.459773429 container remove 931173c35d90ac4d46e813bb3a41aaf031cea517f6130d38fa1a844d1c3ea498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:58:39.234 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:39 smithi189 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-10T06:58:39.234 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:39 smithi189 systemd[1]: Stopped Ceph osd.2 for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:39.234 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 10 06:58:39 smithi189 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@osd.2.service: Consumed 3.881s CPU time. 2024-02-10T06:58:39.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:39.247 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-10T06:58:39.247 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-10T06:58:39.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@rgw.foo.a 2024-02-10T06:58:39.534 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:39 smithi076 systemd[1]: Stopping Ceph rgw.foo.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:39.939 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:39 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a[51017]: 2024-02-10T06:58:39.597+0000 7fb1ce28c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:58:39.939 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:39 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a[51017]: 2024-02-10T06:58:39.597+0000 7fb1dcafdac0 -1 shutting down 2024-02-10T06:58:49.929 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:49 smithi076 bash[61501]: time="2024-02-10T06:58:49Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-10T06:58:49.929 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:49 smithi076 podman[61501]: 2024-02-10 06:58:49.626368347 +0000 UTC m=+10.188603575 container died 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-10T06:58:50.181 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:50 smithi076 podman[61501]: 2024-02-10 06:58:50.120212098 +0000 UTC m=+10.682447403 container cleanup 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-10T06:58:50.181 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:50 smithi076 bash[61501]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a 2024-02-10T06:58:50.459 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:50 smithi076 podman[61539]: 2024-02-10 06:58:50.27148442 +0000 UTC m=+0.638541499 container remove 28ab6ea5bc310f559ff51a1b5639fc6bbfad61d3763b224557e0722afb04fa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-rgw-foo-a, RELEASE=main-290212a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, ceph=True) 2024-02-10T06:58:50.460 INFO:journalctl@ceph.rgw.foo.a.smithi076.stdout:Feb 10 06:58:50 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-10T06:58:50.476 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@rgw.foo.a.service' 2024-02-10T06:58:50.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:50.516 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-10T06:58:50.516 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-10T06:58:50.516 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e8085918-c7e0-11ee-95b8-87774f69a715@prometheus.a 2024-02-10T06:58:50.842 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 systemd[1]: Stopping Ceph prometheus.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:51.132 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T06:58:51.132 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T06:58:51.133 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T06:58:51.133 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T06:58:51.133 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T06:58:51.133 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T06:58:51.133 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T06:58:51.134 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T06:58:51.134 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T06:58:51.134 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T06:58:51.134 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T06:58:51.134 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a[40890]: ts=2024-02-10T06:58:50.842Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T06:58:51.135 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:50 smithi136 podman[44248]: 2024-02-10 06:58:50.852655938 +0000 UTC m=+0.147886366 container died e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:51.386 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:51 smithi136 podman[44248]: 2024-02-10 06:58:51.239043816 +0000 UTC m=+0.534274250 container cleanup e22754816e2166729015d4f888208bf1c50420e13075b004296bb300ccbc213d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:51.386 INFO:journalctl@ceph.prometheus.a.smithi136.stdout:Feb 10 06:58:51 smithi136 bash[44248]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T06:58:51.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e8085918-c7e0-11ee-95b8-87774f69a715@prometheus.a.service' 2024-02-10T06:58:51.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T06:58:51.649 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-10T06:58:51.649 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T06:58:51.836 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:58:52.784 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:52 smithi076 systemd[1]: Stopping Ceph alertmanager.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:53.276 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:52 smithi076 ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a[58151]: ts=2024-02-10T06:58:52.927Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T06:58:53.276 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:52 smithi076 podman[61860]: 2024-02-10 06:58:52.948448193 +0000 UTC m=+0.130013852 container died f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:53.534 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 podman[61860]: 2024-02-10 06:58:53.459100867 +0000 UTC m=+0.640666521 container cleanup f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:53.534 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 bash[61860]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T06:58:54.034 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 podman[61879]: 2024-02-10 06:58:53.652200199 +0000 UTC m=+0.706435816 container remove f00955ad369d606d497817ae43b1904ec2f342540d53e3e5b9893b3e14e63db4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:54.034 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 podman[61879]: 2024-02-10 06:58:53.736365484 +0000 UTC m=+0.790601100 volume remove 746f866c991fcf80c77f8020e304bdbe8eba1dbd486bee968facb77363d08173 2024-02-10T06:58:54.034 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T06:58:54.034 INFO:journalctl@ceph.alertmanager.a.smithi076.stdout:Feb 10 06:58:53 smithi076 systemd[1]: Stopped Ceph alertmanager.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:54.534 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 systemd[1]: Stopping Ceph node-exporter.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:54.534 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 podman[61996]: 2024-02-10 06:58:54.311346504 +0000 UTC m=+0.114303198 container died 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:54.815 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 podman[61996]: 2024-02-10 06:58:54.639403289 +0000 UTC m=+0.442359992 container cleanup 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:54.815 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 bash[61996]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T06:58:55.068 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 podman[62017]: 2024-02-10 06:58:54.815445113 +0000 UTC m=+0.503582533 container remove 57961ca0e9ee8fed2f8fa3804195ef5650ab4b08e8ecb2421d19d6d612c3baf8 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:58:55.069 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:54 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T06:58:55.069 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:55 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-10T06:58:55.069 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:55 smithi076 systemd[1]: Stopped Ceph node-exporter.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:55.069 INFO:journalctl@ceph.node-exporter.a.smithi076.stdout:Feb 10 06:58:55 smithi076 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.a.service: Consumed 1.804s CPU time. 2024-02-10T06:58:55.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T06:58:55.864 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:58:56.886 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:56 smithi136 systemd[1]: Stopping Ceph grafana.a for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:57.275 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:56 smithi136 podman[44587]: 2024-02-10 06:58:56.96438781 +0000 UTC m=+0.122215006 container died 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a, maintainer=Paul Cuzner , distribution-scope=public, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, name=ubi8, vendor=Red Hat, Inc., io.openshift.expose-services=, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0) 2024-02-10T06:58:57.275 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 podman[44587]: 2024-02-10 06:58:57.27527718 +0000 UTC m=+0.433104381 container cleanup 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, com.redhat.component=ubi8-container, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.openshift.tags=base rhel8, version=8.5, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, vcs-type=git, vendor=Red Hat, Inc., io.buildah.version=1.32.0) 2024-02-10T06:58:57.550 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 bash[44587]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a 2024-02-10T06:58:57.550 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 podman[44606]: 2024-02-10 06:58:57.468071512 +0000 UTC m=+0.506179339 container remove 182a4c1deb30cc96d90aa6d55d1372173e5f7da360a03b2ac1807e88050a372e (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , vendor=Red Hat, Inc., architecture=x86_64, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, distribution-scope=public, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, version=8.5, io.openshift.tags=base rhel8, release=239.1651231664, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container) 2024-02-10T06:58:57.550 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T06:58:57.885 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:57 smithi136 systemd[1]: Stopping Ceph node-exporter.b for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:58:57.886 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-10T06:58:57.886 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 systemd[1]: Stopped Ceph grafana.a for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:57.886 INFO:journalctl@ceph.grafana.a.smithi136.stdout:Feb 10 06:58:57 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@grafana.a.service: Consumed 3.177s CPU time. 2024-02-10T06:58:58.280 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 podman[44724]: 2024-02-10 06:58:58.026781425 +0000 UTC m=+0.120001474 container died a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:58:58.606 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 podman[44724]: 2024-02-10 06:58:58.378689941 +0000 UTC m=+0.471909995 container cleanup a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:58:58.606 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 bash[44724]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T06:58:58.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 podman[44742]: 2024-02-10 06:58:58.646809947 +0000 UTC m=+0.624940964 container remove a2e0347f9184056a740f8de883dce93c4a216ce9f6bceac4d45fd680f746c8ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:58:58.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-10T06:58:58.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-10T06:58:58.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 systemd[1]: Stopped Ceph node-exporter.b for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:58:58.886 INFO:journalctl@ceph.node-exporter.b.smithi136.stdout:Feb 10 06:58:58 smithi136 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.b.service: Consumed 1.788s CPU time. 2024-02-10T06:58:59.536 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T06:58:59.725 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:59:00.671 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:00 smithi189 systemd[1]: Stopping Ceph node-exporter.c for e8085918-c7e0-11ee-95b8-87774f69a715... 2024-02-10T06:59:01.048 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:00 smithi189 podman[45472]: 2024-02-10 06:59:00.729572108 +0000 UTC m=+0.115780457 container died 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:59:01.048 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 podman[45472]: 2024-02-10 06:59:01.048508266 +0000 UTC m=+0.434716613 container cleanup 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:59:01.421 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 bash[45472]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c 2024-02-10T06:59:01.421 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 podman[45493]: 2024-02-10 06:59:01.26629768 +0000 UTC m=+0.537484764 container remove 5b2c6563f7eb8923350a285a7af7623ff2306f8d6a8f0f833360c2cd95adfd74 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e8085918-c7e0-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T06:59:01.421 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-02-10T06:59:01.898 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.c.service: Failed with result 'exit-code'. 2024-02-10T06:59:01.899 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 systemd[1]: Stopped Ceph node-exporter.c for e8085918-c7e0-11ee-95b8-87774f69a715. 2024-02-10T06:59:01.899 INFO:journalctl@ceph.node-exporter.c.smithi189.stdout:Feb 10 06:59:01 smithi189 systemd[1]: ceph-e8085918-c7e0-11ee-95b8-87774f69a715@node-exporter.c.service: Consumed 1.788s CPU time. 2024-02-10T06:59:02.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:59:02.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:59:02.176 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:59:02.206 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-10T06:59:02.208 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi076/crash 2024-02-10T06:59:02.209 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash -- . 2024-02-10T06:59:02.260 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T06:59:02.261 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-02-10T06:59:02.262 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi136/crash 2024-02-10T06:59:02.263 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash -- . 2024-02-10T06:59:02.315 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T06:59:02.315 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-02-10T06:59:02.317 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi189/crash 2024-02-10T06:59:02.318 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash -- . 2024-02-10T06:59:02.358 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T06:59:02.358 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-02-10T06:59:02.359 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-10T06:59:02.360 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T06:59:02.391 INFO:teuthology.orchestra.run.smithi076.stdout:2024-02-10T06:55:04.902897+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T06:59:02.392 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-10T06:59:02.392 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[SEC\]' /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T06:59:02.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]' /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T06:59:02.532 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[WRN\]' /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T06:59:02.601 INFO:teuthology.orchestra.run.smithi076.stdout:2024-02-10T06:55:04.902897+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T06:59:02.602 INFO:tasks.cephadm:Compressing logs... 2024-02-10T06:59:02.602 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T06:59:02.644 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T06:59:02.646 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T06:59:02.669 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:: No such file or directory 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.a.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.y.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.0.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.y.log: 57.8% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T06:59:02.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log 2024-02-10T06:59:02.671 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T06:59:02.672 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T06:59:02.672 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T06:59:02.672 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log: 82.6% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T06:59:02.672 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log: 84.9% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T06:59:02.673 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T06:59:02.673 INFO:teuthology.orchestra.run.smithi189.stderr:: No such file or directory 2024-02-10T06:59:02.673 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T06:59:02.673 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.z.log 2024-02-10T06:59:02.674 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log: /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.2.log 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr: 81.4% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr: 89.6% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.c.log 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr: 84.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T06:59:02.678 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log: /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.c.log: 90.2% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.z.log.gz 2024-02-10T06:59:02.679 INFO:teuthology.orchestra.run.smithi189.stderr: 90.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T06:59:02.679 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.1.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.x.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log: /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.x.log: /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log: 89.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr: 81.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.b.log 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log: 84.4% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T06:59:02.680 INFO:teuthology.orchestra.run.smithi136.stderr: 81.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T06:59:02.681 INFO:teuthology.orchestra.run.smithi076.stderr: 90.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T06:59:02.681 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.b.log: 90.5% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T06:59:02.681 INFO:teuthology.orchestra.run.smithi136.stderr: 90.2% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.x.log.gz 2024-02-10T06:59:02.732 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.b.log.gz 2024-02-10T06:59:02.733 INFO:teuthology.orchestra.run.smithi189.stderr: 92.3% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.c.log.gz 2024-02-10T06:59:02.736 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mgr.y.log.gz 2024-02-10T06:59:02.847 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-mon.a.log.gz 2024-02-10T06:59:02.923 INFO:teuthology.orchestra.run.smithi076.stderr: 94.3% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.0.log.gz 2024-02-10T06:59:02.925 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-02-10T06:59:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m0.269s 2024-02-10T06:59:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m0.502s 2024-02-10T06:59:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.037s 2024-02-10T06:59:02.933 INFO:teuthology.orchestra.run.smithi189.stderr: 94.3% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.2.log.gz 2024-02-10T06:59:02.935 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-02-10T06:59:02.935 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m0.275s 2024-02-10T06:59:02.935 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m0.330s 2024-02-10T06:59:02.935 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.029s 2024-02-10T06:59:02.974 INFO:teuthology.orchestra.run.smithi136.stderr: 94.3% -- replaced with /var/log/ceph/e8085918-c7e0-11ee-95b8-87774f69a715/ceph-osd.1.log.gz 2024-02-10T06:59:02.976 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-02-10T06:59:02.977 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m0.316s 2024-02-10T06:59:02.977 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m0.360s 2024-02-10T06:59:02.977 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.041s 2024-02-10T06:59:02.977 INFO:tasks.cephadm:Archiving logs... 2024-02-10T06:59:02.977 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi076/log 2024-02-10T06:59:02.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T06:59:03.103 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi136/log 2024-02-10T06:59:03.104 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T06:59:03.195 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi189/log 2024-02-10T06:59:03.196 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T06:59:03.277 INFO:tasks.cephadm:Removing cluster... 2024-02-10T06:59:03.277 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force 2024-02-10T06:59:03.456 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:59:03.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force 2024-02-10T06:59:03.749 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:59:03.865 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e8085918-c7e0-11ee-95b8-87774f69a715 --force 2024-02-10T06:59:04.052 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: e8085918-c7e0-11ee-95b8-87774f69a715 2024-02-10T06:59:04.167 INFO:tasks.cephadm:Removing cephadm ... 2024-02-10T06:59:04.167 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T06:59:04.183 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T06:59:04.198 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T06:59:04.213 INFO:tasks.cephadm:Teardown complete 2024-02-10T06:59:04.213 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-10T06:59:04.222 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi076:/dev/vg_nvme/lv_1... 2024-02-10T06:59:04.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo nvme disconnect -n lv_1 2024-02-10T06:59:04.372 INFO:teuthology.orchestra.run.smithi076.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T06:59:04.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:04.374 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi076:/dev/vg_nvme/lv_2... 2024-02-10T06:59:04.374 DEBUG:teuthology.orchestra.run.smithi076:> sudo nvme disconnect -n lv_2 2024-02-10T06:59:04.524 INFO:teuthology.orchestra.run.smithi076.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T06:59:04.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:04.526 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi076:/dev/vg_nvme/lv_3... 2024-02-10T06:59:04.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo nvme disconnect -n lv_3 2024-02-10T06:59:04.693 INFO:teuthology.orchestra.run.smithi076.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T06:59:04.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:04.695 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi076:/dev/vg_nvme/lv_4... 2024-02-10T06:59:04.695 DEBUG:teuthology.orchestra.run.smithi076:> sudo nvme disconnect -n lv_4 2024-02-10T06:59:04.840 INFO:teuthology.orchestra.run.smithi076.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T06:59:04.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:04.842 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-10T06:59:04.842 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/scratch_devs 2024-02-10T06:59:04.869 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi136:/dev/vg_nvme/lv_1... 2024-02-10T06:59:04.869 DEBUG:teuthology.orchestra.run.smithi136:> sudo nvme disconnect -n lv_1 2024-02-10T06:59:05.019 INFO:teuthology.orchestra.run.smithi136.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T06:59:05.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.021 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi136:/dev/vg_nvme/lv_2... 2024-02-10T06:59:05.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo nvme disconnect -n lv_2 2024-02-10T06:59:05.160 INFO:teuthology.orchestra.run.smithi136.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T06:59:05.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.162 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi136:/dev/vg_nvme/lv_3... 2024-02-10T06:59:05.162 DEBUG:teuthology.orchestra.run.smithi136:> sudo nvme disconnect -n lv_3 2024-02-10T06:59:05.312 INFO:teuthology.orchestra.run.smithi136.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T06:59:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.314 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi136:/dev/vg_nvme/lv_4... 2024-02-10T06:59:05.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo nvme disconnect -n lv_4 2024-02-10T06:59:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T06:59:05.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.451 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-02-10T06:59:05.451 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/scratch_devs 2024-02-10T06:59:05.484 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_1... 2024-02-10T06:59:05.484 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_1 2024-02-10T06:59:05.635 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T06:59:05.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.637 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_2... 2024-02-10T06:59:05.638 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_2 2024-02-10T06:59:05.774 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T06:59:05.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.776 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_3... 2024-02-10T06:59:05.776 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_3 2024-02-10T06:59:05.916 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T06:59:05.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:05.918 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_4... 2024-02-10T06:59:05.918 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_4 2024-02-10T06:59:06.057 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T06:59:06.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:06.059 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-10T06:59:06.059 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/scratch_devs 2024-02-10T06:59:06.087 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-10T06:59:06.100 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-10T06:59:06.100 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:59:06.102 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:59:06.105 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:59:06.115 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +947us[ +947us] +/- 92ms 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -971us[ -973us] +/- 53ms 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +646us[ +646us] +/- 67ms 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:59:06.119 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +931us[ +952us] +/- 92ms 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -980us[ -960us] +/- 53ms 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +633us[ +653us] +/- 67ms 2024-02-10T06:59:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:59:06.142 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:59:06.146 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:59:06.146 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-02-10T06:59:06.146 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +922us[ +939us] +/- 92ms 2024-02-10T06:59:06.146 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -981us[ -964us] +/- 53ms 2024-02-10T06:59:06.146 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +616us[ +633us] +/- 67ms 2024-02-10T06:59:06.147 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:59:06.148 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-10T06:59:06.157 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-10T06:59:06.157 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-10T06:59:06.165 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T06:59:06.238 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T06:59:06.313 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T06:59:06.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:59:06.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:06.415 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:59:06.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:06.448 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:59:06.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:06.478 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-10T06:59:06.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-10T06:59:06.496 INFO:teuthology.task.internal:Duration was 689.070653 seconds 2024-02-10T06:59:06.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-10T06:59:06.507 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-10T06:59:06.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T06:59:06.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T06:59:06.512 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T06:59:06.550 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:59:06.555 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:59:06.562 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:59:06.926 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-10T06:59:06.926 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-02-10T06:59:06.926 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T06:59:06.945 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-02-10T06:59:06.946 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T06:59:06.974 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-02-10T06:59:06.974 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T06:59:06.996 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-10T06:59:06.996 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T06:59:06.999 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T06:59:07.016 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T06:59:07.065 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-10T06:59:07.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T06:59:07.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T06:59:07.087 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T06:59:07.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-10T06:59:07.266 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-10T06:59:07.266 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T06:59:07.293 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T06:59:07.320 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T06:59:07.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-10T06:59:07.358 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.360 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.362 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.383 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-02-10T06:59:07.385 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-02-10T06:59:07.414 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-02-10T06:59:07.435 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:07.460 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:07.474 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T06:59:07.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:59:07.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-10T06:59:07.499 INFO:teuthology.task.internal:Transferring archived files... 2024-02-10T06:59:07.500 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi076 2024-02-10T06:59:07.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T06:59:07.542 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi136 2024-02-10T06:59:07.542 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T06:59:07.578 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166/remote/smithi189 2024-02-10T06:59:07.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T06:59:07.613 INFO:teuthology.task.internal:Removing archive directory... 2024-02-10T06:59:07.613 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T06:59:07.615 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T06:59:07.617 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T06:59:07.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-10T06:59:07.672 INFO:teuthology.task.internal:Not uploading archives. 2024-02-10T06:59:07.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-10T06:59:07.731 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-10T06:59:07.732 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T06:59:07.734 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T06:59:07.736 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T06:59:07.746 INFO:teuthology.orchestra.run.smithi076.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 06:59 /home/ubuntu/cephtest 2024-02-10T06:59:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 06:59 /home/ubuntu/cephtest 2024-02-10T06:59:07.750 INFO:teuthology.orchestra.run.smithi189.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 06:59 /home/ubuntu/cephtest 2024-02-10T06:59:07.751 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-10T06:59:07.761 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-10T06:59:07.792 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T06:59:07.819 DEBUG:teuthology.nuke:shortname: smithi076 2024-02-10T06:59:07.819 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T06:59:07.842 DEBUG:teuthology.nuke:shortname: smithi136 2024-02-10T06:59:07.843 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T06:59:07.847 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.528159', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKb1H6XlSqlcBBqJgKD9uy3eDN5+yr0+g26Xdf2Q0GFF0ttGanEfF3wuF59w++cjDVofVxg5yp1QMG5NXCPCiog='} 2024-02-10T06:59:07.865 DEBUG:teuthology.nuke:shortname: smithi189 2024-02-10T06:59:07.865 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T06:59:07.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.525541', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAh9RQHm7p8CKgkz/ikf6UdICueZGSo11pgyncO2VRdjXA7xaBoSYqQvpxUBsuU79C8IM9bmKCCLTGAFvj0YHs='} 2024-02-10T06:59:07.891 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555166', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:37:52.526963', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGqbhrGVNQZB/4hdAQAJu6Fc2hWkbfn+jUl3uQWkhWV/ClgsrebZWD+Bnb8WDR5sFPziPWMGABPVqlsn8NU1jw='} 2024-02-10T06:59:07.900 INFO:teuthology.orchestra.console.smithi076:Power off 2024-02-10T06:59:07.900 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T06:59:07.923 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-02-10T06:59:07.924 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:07.941 INFO:teuthology.orchestra.console.smithi136:Power off 2024-02-10T06:59:07.941 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T06:59:07.950 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-02-10T06:59:07.962 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-02-10T06:59:07.962 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:07.977 INFO:teuthology.orchestra.console.smithi189:Power off 2024-02-10T06:59:07.977 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T06:59:07.987 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-02-10T06:59:07.999 DEBUG:teuthology.orchestra.console.smithi189:power off output: Chassis Power Control: Down/Off 2024-02-10T06:59:07.999 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:08.020 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-02-10T06:59:11.952 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:11.989 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:12.021 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:12.062 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-02-10T06:59:12.098 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-02-10T06:59:12.131 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-02-10T06:59:16.064 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:16.099 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:16.132 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:16.173 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-02-10T06:59:16.209 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-02-10T06:59:16.241 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is off 2024-02-10T06:59:16.275 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-02-10T06:59:16.342 INFO:teuthology.orchestra.console.smithi189:Power off completed 2024-02-10T06:59:20.214 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T06:59:20.324 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-02-10T06:59:20.425 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-02-10T06:59:20.529 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 689.0706534385681 failure_reason: '"2024-02-10T06:55:04.902897+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_yuriw@teuthology success: false 2024-02-10T06:59:20.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T06:59:20.777 INFO:teuthology.run:FAIL